[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:07:55 dist-test-slave-zpfg krb5kdc[27634](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:07:55 dist-test-slave-zpfg krb5kdc[27634](info): set up 2 sockets
Aug 12 03:07:55 dist-test-slave-zpfg krb5kdc[27634](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 03:07:56.435305 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:07:56 dist-test-slave-zpfg krb5kdc[27634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:07:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:07:56.576076 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:45331
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:40739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:45331
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:56.694036 27647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:56.694414 27647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:56.694531 27647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:56.698670 27647 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:56.698786 27647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:56.698822 27647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:56.698865 27647 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:56.698899 27647 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:56.705555 27647 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:40739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:45331
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:45331
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:56.706902 27647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:56.708068 27647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:56.718730 27655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:56.718763 27652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:56.718760 27653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:56.718986 27647 server_base.cc:1034] running on GCE node
I20260812 03:07:56.719604 27647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:56.720810 27647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:56.722031 27647 hybrid_clock.cc:648] HybridClock initialized: now 1786504076721978 us; error 117 us; skew 500 ppm
Aug 12 03:07:56 dist-test-slave-zpfg krb5kdc[27634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504076, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:56.725432 27647 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:07:56.726634 27647 webserver.cc:460] Webserver started at http://127.26.250.190:46189/ using document root <none> and password file <none>
I20260812 03:07:56.727324 27647 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:56.727458 27647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:56.727735 27647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:56.730155 27647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "25bca576276749d189a086e90c319abd"
format_stamp: "Formatted at 2026-08-12 03:07:56 on dist-test-slave-zpfg"
server_key: "c31444741918585abe49683c386d3d2e"
server_key_iv: "4f4b2c00470c151893f03b4ba3986fa5"
server_key_version: "encryptionkey@0"
I20260812 03:07:56.730902 27647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "25bca576276749d189a086e90c319abd"
format_stamp: "Formatted at 2026-08-12 03:07:56 on dist-test-slave-zpfg"
server_key: "c31444741918585abe49683c386d3d2e"
server_key_iv: "4f4b2c00470c151893f03b4ba3986fa5"
server_key_version: "encryptionkey@0"
I20260812 03:07:56.735357 27647 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:07:56.738085 27662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:56.739586 27647 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 03:07:56.739764 27647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "25bca576276749d189a086e90c319abd"
format_stamp: "Formatted at 2026-08-12 03:07:56 on dist-test-slave-zpfg"
server_key: "c31444741918585abe49683c386d3d2e"
server_key_iv: "4f4b2c00470c151893f03b4ba3986fa5"
server_key_version: "encryptionkey@0"
I20260812 03:07:56.739929 27647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:56.759229 27647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:56.762812 27647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:56.763049 27647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:56.771181 27647 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:45331
I20260812 03:07:56.771212 27716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:45331 every 8 connection(s)
I20260812 03:07:56.772197 27647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:56.775173 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 27647
I20260812 03:07:56.775383 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:07:56.776033 27717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:56.777467 27626 external_mini_cluster.cc:1428] Setting key e93e6e5e333272709463421612471704
I20260812 03:07:56.784277 27717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25bca576276749d189a086e90c319abd: Bootstrap starting.
I20260812 03:07:56.786854 27717 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25bca576276749d189a086e90c319abd: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:56.787696 27717 log.cc:824] T 00000000000000000000000000000000 P 25bca576276749d189a086e90c319abd: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:56.789745 27717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25bca576276749d189a086e90c319abd: No bootstrap required, opened a new log
I20260812 03:07:56.793432 27717 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25bca576276749d189a086e90c319abd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25bca576276749d189a086e90c319abd" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45331 } }
I20260812 03:07:56.793715 27717 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25bca576276749d189a086e90c319abd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:56.793795 27717 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25bca576276749d189a086e90c319abd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25bca576276749d189a086e90c319abd, State: Initialized, Role: FOLLOWER
I20260812 03:07:56.794309 27717 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25bca576276749d189a086e90c319abd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25bca576276749d189a086e90c319abd" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45331 } }
I20260812 03:07:56.794492 27717 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25bca576276749d189a086e90c319abd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:07:56.794597 27717 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25bca576276749d189a086e90c319abd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:07:56.794716 27717 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25bca576276749d189a086e90c319abd [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:56.795881 27717 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25bca576276749d189a086e90c319abd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25bca576276749d189a086e90c319abd" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45331 } }
I20260812 03:07:56.796262 27717 leader_election.cc:302] T 00000000000000000000000000000000 P 25bca576276749d189a086e90c319abd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25bca576276749d189a086e90c319abd; no voters:
I20260812 03:07:56.796597 27717 leader_election.cc:288] T 00000000000000000000000000000000 P 25bca576276749d189a086e90c319abd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:07:56.796744 27722 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25bca576276749d189a086e90c319abd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:56.797045 27722 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25bca576276749d189a086e90c319abd [term 1 LEADER]: Becoming Leader. State: Replica: 25bca576276749d189a086e90c319abd, State: Running, Role: LEADER
Aug 12 03:07:56 dist-test-slave-zpfg krb5kdc[27634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:07:56.797472 27722 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25bca576276749d189a086e90c319abd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25bca576276749d189a086e90c319abd" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45331 } }
I20260812 03:07:56.798049 27717 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25bca576276749d189a086e90c319abd [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:56.799532 27724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25bca576276749d189a086e90c319abd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25bca576276749d189a086e90c319abd. Latest consensus state: current_term: 1 leader_uuid: "25bca576276749d189a086e90c319abd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25bca576276749d189a086e90c319abd" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45331 } } }
I20260812 03:07:56.799531 27723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25bca576276749d189a086e90c319abd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25bca576276749d189a086e90c319abd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25bca576276749d189a086e90c319abd" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45331 } } }
I20260812 03:07:56.799695 27724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25bca576276749d189a086e90c319abd [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:56.799695 27723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25bca576276749d189a086e90c319abd [sys.catalog]: This master's current role is: LEADER
W20260812 03:07:56.803714 27738 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 25bca576276749d189a086e90c319abd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:07:56.803814 27738 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:07:56.803884 27732 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:56.804793 27732 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:56.805575 27720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:56.784275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34122 (local address 127.26.250.190:45331)
0812 03:07:56.784763 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:56.784782 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:56.784831 (+ 49us) server_negotiation.cc:407] Connection header received
0812 03:07:56.787460 (+ 2629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:56.787479 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:56.787871 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:56.788240 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:56.790610 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:56.791633 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:56.792890 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:56.793392 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:56.793589 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:56.798639 (+ 5050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:56.798649 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:56.798668 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:56.798707 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:56.801260 (+ 2553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:56.802025 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:56.802040 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:56.802044 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:56.802158 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:56.802777 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:56.802780 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:56.802781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:56.803283 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:56.803530 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:56.804197 (+ 667us) server_negotiation.cc:299] Negotiation successful
0812 03:07:56.804490 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":143,"threads_started":1}
I20260812 03:07:56.811893 27732 catalog_manager.cc:1347] Generated new cluster ID: a4fbf8221b144f1dad1ad705632a6149
I20260812 03:07:56.811990 27732 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:56.820003 27732 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:56.821135 27732 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:56.830224 27732 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25bca576276749d189a086e90c319abd: Generated new TSK 0
I20260812 03:07:56.831310 27732 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.250.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
I20260812 03:07:56.878373 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:0
--local_ip_for_outbound_sockets=127.26.250.129
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:45331
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:40739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:57.006327 27744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:57.006754 27744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:57.006870 27744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:57.011256 27744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:57.011366 27744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:57.011464 27744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:07:57.018309 27744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:40739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:45331
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:57.019660 27744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:57.020893 27744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:07:57.033172 27744 server_base.cc:1034] running on GCE node
W20260812 03:07:57.033114 27751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:57.033097 27750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:57.033107 27753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:57.033907 27744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:57.034670 27744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:57.035887 27744 hybrid_clock.cc:648] HybridClock initialized: now 1786504077035853 us; error 54 us; skew 500 ppm
Aug 12 03:07:57 dist-test-slave-zpfg krb5kdc[27634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504077, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:57.039155 27744 init.cc:377] Logged in from keytab as kudu/127.26.250.129@KRBTEST.COM (short username kudu)
I20260812 03:07:57.040071 27744 webserver.cc:460] Webserver started at http://127.26.250.129:46243/ using document root <none> and password file <none>
I20260812 03:07:57.040624 27744 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:57.040686 27744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:57.040942 27744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:57.043326 27744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-0/data/instance:
uuid: "339365231a6f47bca86a7bc68d67da4c"
format_stamp: "Formatted at 2026-08-12 03:07:57 on dist-test-slave-zpfg"
server_key: "c4603f840ead1ff7ca75f2d44c5654f0"
server_key_iv: "0d181239686ba94db56a9d074c522ed4"
server_key_version: "encryptionkey@0"
I20260812 03:07:57.044155 27744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance:
uuid: "339365231a6f47bca86a7bc68d67da4c"
format_stamp: "Formatted at 2026-08-12 03:07:57 on dist-test-slave-zpfg"
server_key: "c4603f840ead1ff7ca75f2d44c5654f0"
server_key_iv: "0d181239686ba94db56a9d074c522ed4"
server_key_version: "encryptionkey@0"
I20260812 03:07:57.048662 27744 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:07:57.051924 27761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:57.053274 27744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:07:57.053465 27744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "339365231a6f47bca86a7bc68d67da4c"
format_stamp: "Formatted at 2026-08-12 03:07:57 on dist-test-slave-zpfg"
server_key: "c4603f840ead1ff7ca75f2d44c5654f0"
server_key_iv: "0d181239686ba94db56a9d074c522ed4"
server_key_version: "encryptionkey@0"
I20260812 03:07:57.053611 27744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:57.077265 27744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:57.081046 27744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:57.081272 27744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:57.082007 27744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:07:57.082998 27744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:07:57.083071 27744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:57.083142 27744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:07:57.083184 27744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:57.093166 27744 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:45145
I20260812 03:07:57.093222 27882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:45145 every 8 connection(s)
I20260812 03:07:57.094198 27744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:07:57.097352 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 27744
I20260812 03:07:57.097474 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance
I20260812 03:07:57.097762 27626 external_mini_cluster.cc:1428] Setting key ee4a15ae248735dde05fd8fe667c7eda
Aug 12 03:07:57 dist-test-slave-zpfg krb5kdc[27634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504077, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.250.130@KRBTEST.COM; defaulting to no policy
I20260812 03:07:57.111117 27720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:57.096327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:34023 (local address 127.26.250.190:45331)
0812 03:07:57.096474 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:57.096478 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:57.097580 (+ 1102us) server_negotiation.cc:407] Connection header received
0812 03:07:57.099476 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:57.099480 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:57.099533 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:57.099629 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:57.101446 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:57.101974 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:57.103267 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:57.103542 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:57.103661 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:57.106649 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:57.106653 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:57.106656 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:57.106690 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:57.108503 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:57.109169 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:57.109173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:57.109175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:57.109270 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:57.109675 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:57.109678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:57.109680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:57.109971 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:57.110106 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:57.110757 (+ 651us) server_negotiation.cc:299] Negotiation successful
0812 03:07:57.110929 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.130@KRBTEST.COM" created.
I20260812 03:07:57.112248 27885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:57.096798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:45331 (local address 127.26.250.129:34023)
0812 03:07:57.097392 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:57.097460 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:57.099222 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:57.099835 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:57.099845 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:57.100489 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:57.101266 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:57.101300 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:57.102120 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:57.102128 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:57.103129 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:57.103139 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:57.103710 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:57.103715 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:57.103901 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:57.104556 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:57.104574 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:57.106461 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:57.108655 (+ 2194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:57.108662 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:57.108681 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:57.109046 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:57.109378 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:57.109381 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:57.109383 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:57.109563 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:57.110106 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:57.110114 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:57.110497 (+ 383us) client_negotiation.cc:769] Sending connection context
0812 03:07:57.110699 (+ 202us) client_negotiation.cc:240] Negotiation successful
0812 03:07:57.111085 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":123,"threads_started":1}
I20260812 03:07:57.113965 27883 heartbeater.cc:347] Connected to a master server at 127.26.250.190:45331
I20260812 03:07:57.114286 27883 heartbeater.cc:464] Registering TS with master...
I20260812 03:07:57.115069 27883 heartbeater.cc:511] Master 127.26.250.190:45331 requested a full tablet report, sending...
I20260812 03:07:57.117019 27680 ts_manager.cc:194] Registered new tserver with Master: 339365231a6f47bca86a7bc68d67da4c (127.26.250.129:45145)
I20260812 03:07:57.118311 27680 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.129@KRBTEST.COM'} at 127.26.250.129:34023
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
I20260812 03:07:57.135722 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:0
--local_ip_for_outbound_sockets=127.26.250.130
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:45331
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:40739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:57.258330 27889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:57.258697 27889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:57.258810 27889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:57.263046 27889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:57.263157 27889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:57.263257 27889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:07:57.269182 27889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:40739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:45331
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:57.270471 27889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:57.271596 27889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:57.282338 27897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:57.282390 27894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:57.282438 27895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:57.282791 27889 server_base.cc:1034] running on GCE node
I20260812 03:07:57.283442 27889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:57.284339 27889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:57.285568 27889 hybrid_clock.cc:648] HybridClock initialized: now 1786504077285531 us; error 57 us; skew 500 ppm
Aug 12 03:07:57 dist-test-slave-zpfg krb5kdc[27634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504077, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:57.289032 27889 init.cc:377] Logged in from keytab as kudu/127.26.250.130@KRBTEST.COM (short username kudu)
I20260812 03:07:57.290145 27889 webserver.cc:460] Webserver started at http://127.26.250.130:40673/ using document root <none> and password file <none>
I20260812 03:07:57.290884 27889 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:57.290990 27889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:57.291258 27889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:57.293622 27889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-1/data/instance:
uuid: "404e40a514794a55879c24b6a163a355"
format_stamp: "Formatted at 2026-08-12 03:07:57 on dist-test-slave-zpfg"
server_key: "639fe235abc8dee83ca9178ecaaba1e6"
server_key_iv: "1ff089e212ca2ec90250e8712ad068c4"
server_key_version: "encryptionkey@0"
I20260812 03:07:57.294255 27889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance:
uuid: "404e40a514794a55879c24b6a163a355"
format_stamp: "Formatted at 2026-08-12 03:07:57 on dist-test-slave-zpfg"
server_key: "639fe235abc8dee83ca9178ecaaba1e6"
server_key_iv: "1ff089e212ca2ec90250e8712ad068c4"
server_key_version: "encryptionkey@0"
I20260812 03:07:57.298826 27889 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:07:57.302601 27904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:57.304366 27889 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:07:57.304563 27889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "404e40a514794a55879c24b6a163a355"
format_stamp: "Formatted at 2026-08-12 03:07:57 on dist-test-slave-zpfg"
server_key: "639fe235abc8dee83ca9178ecaaba1e6"
server_key_iv: "1ff089e212ca2ec90250e8712ad068c4"
server_key_version: "encryptionkey@0"
I20260812 03:07:57.304714 27889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:57.326097 27889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:57.329603 27889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:57.329830 27889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:57.330583 27889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:07:57.331488 27889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:07:57.331570 27889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:57.331641 27889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:07:57.331692 27889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:57.341971 27889 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:40757
I20260812 03:07:57.341995 28023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:40757 every 8 connection(s)
I20260812 03:07:57.343032 27889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:07:57.344484 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 27889
I20260812 03:07:57.344609 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance
I20260812 03:07:57.344923 27626 external_mini_cluster.cc:1428] Setting key 49b5c81f81e2f4c216833da4e0818bcc
Aug 12 03:07:57 dist-test-slave-zpfg krb5kdc[27634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504077, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.250.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.131@KRBTEST.COM" created.
I20260812 03:07:57.361618 27720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:57.345632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:48425 (local address 127.26.250.190:45331)
0812 03:07:57.345800 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:57.345805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:57.346887 (+ 1082us) server_negotiation.cc:407] Connection header received
0812 03:07:57.348261 (+ 1374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:57.348268 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:57.348343 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:57.348460 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:57.349743 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:57.350723 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:57.352710 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:57.352958 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:57.353067 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:57.356722 (+ 3655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:57.356726 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:57.356729 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:57.356765 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:57.358694 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:57.359369 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:57.359373 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:57.359375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:57.359462 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:57.360032 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:57.360036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:57.360038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:57.360349 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:57.360463 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:57.361359 (+ 896us) server_negotiation.cc:299] Negotiation successful
0812 03:07:57.361486 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 03:07:57.363161 28026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:57.346126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:45331 (local address 127.26.250.130:48425)
0812 03:07:57.346708 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:57.346768 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:57.348024 (+ 1256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:57.348637 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:57.348644 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:57.349026 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:57.349569 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:57.349583 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:57.350893 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:57.350900 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:57.352544 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:57.352560 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:57.353160 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:57.353167 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:57.353445 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:57.354214 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:57.354242 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:57.356496 (+ 2254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:57.358826 (+ 2330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:57.358831 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:57.358846 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:57.359226 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:57.359596 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:57.359599 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:57.359600 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:57.359913 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:57.360475 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:57.360507 (+ 32us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:57.360938 (+ 431us) client_negotiation.cc:769] Sending connection context
0812 03:07:57.361336 (+ 398us) client_negotiation.cc:240] Negotiation successful
0812 03:07:57.361798 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":159,"threads_started":1}
I20260812 03:07:57.364646 28024 heartbeater.cc:347] Connected to a master server at 127.26.250.190:45331
I20260812 03:07:57.364943 28024 heartbeater.cc:464] Registering TS with master...
I20260812 03:07:57.365594 28024 heartbeater.cc:511] Master 127.26.250.190:45331 requested a full tablet report, sending...
I20260812 03:07:57.366756 27680 ts_manager.cc:194] Registered new tserver with Master: 404e40a514794a55879c24b6a163a355 (127.26.250.130:40757)
I20260812 03:07:57.367364 27680 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.130@KRBTEST.COM'} at 127.26.250.130:48425
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
I20260812 03:07:57.381929 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:0
--local_ip_for_outbound_sockets=127.26.250.131
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:45331
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:40739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:57.509173 28029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:57.509548 28029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:57.509637 28029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:57.513967 28029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:57.514042 28029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:57.514138 28029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:07:57.519968 28029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:40739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:45331
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:57.521266 28029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:57.522481 28029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:57.533587 28037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:57.533563 28039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:57.533609 28035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:57.533973 28029 server_base.cc:1034] running on GCE node
I20260812 03:07:57.534549 28029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:57.535291 28029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:57.536510 28029 hybrid_clock.cc:648] HybridClock initialized: now 1786504077536496 us; error 33 us; skew 500 ppm
Aug 12 03:07:57 dist-test-slave-zpfg krb5kdc[27634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504077, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:57.539831 28029 init.cc:377] Logged in from keytab as kudu/127.26.250.131@KRBTEST.COM (short username kudu)
I20260812 03:07:57.540889 28029 webserver.cc:460] Webserver started at http://127.26.250.131:35223/ using document root <none> and password file <none>
I20260812 03:07:57.541533 28029 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:57.541632 28029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:57.541887 28029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:57.544260 28029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-2/data/instance:
uuid: "7f6b1580daaa49c9a476cea66507e69a"
format_stamp: "Formatted at 2026-08-12 03:07:57 on dist-test-slave-zpfg"
server_key: "9180b9a4403fd8070a5270d175c5a671"
server_key_iv: "34379a6b46f80ae0d194c1d64c7a81eb"
server_key_version: "encryptionkey@0"
I20260812 03:07:57.544940 28029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance:
uuid: "7f6b1580daaa49c9a476cea66507e69a"
format_stamp: "Formatted at 2026-08-12 03:07:57 on dist-test-slave-zpfg"
server_key: "9180b9a4403fd8070a5270d175c5a671"
server_key_iv: "34379a6b46f80ae0d194c1d64c7a81eb"
server_key_version: "encryptionkey@0"
I20260812 03:07:57.549435 28029 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260812 03:07:57.552585 28046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:57.554059 28029 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:07:57.554239 28029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "7f6b1580daaa49c9a476cea66507e69a"
format_stamp: "Formatted at 2026-08-12 03:07:57 on dist-test-slave-zpfg"
server_key: "9180b9a4403fd8070a5270d175c5a671"
server_key_iv: "34379a6b46f80ae0d194c1d64c7a81eb"
server_key_version: "encryptionkey@0"
I20260812 03:07:57.554409 28029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:57.570755 28029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:57.574687 28029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:57.574923 28029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:57.575645 28029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:07:57.576603 28029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:07:57.576681 28029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:57.576761 28029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:07:57.576805 28029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:57.587338 28029 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:42069
I20260812 03:07:57.587371 28168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:42069 every 8 connection(s)
I20260812 03:07:57.588310 28029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:07:57.590505 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 28029
I20260812 03:07:57.590631 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance
I20260812 03:07:57.590935 27626 external_mini_cluster.cc:1428] Setting key bbaa938e6a15f22d20785afb5fef8c5b
Aug 12 03:07:57 dist-test-slave-zpfg krb5kdc[27634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504077, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:07:57.604589 27720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:57.590381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:60177 (local address 127.26.250.190:45331)
0812 03:07:57.590514 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:57.590518 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:57.591748 (+ 1230us) server_negotiation.cc:407] Connection header received
0812 03:07:57.593323 (+ 1575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:57.593327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:57.593395 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:57.593509 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:57.594836 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:57.595430 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:57.596765 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:57.597013 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:57.597098 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:57.600133 (+ 3035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:57.600138 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:57.600141 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:57.600181 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:57.602113 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:57.602881 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:57.602885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:57.602886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:57.602974 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:57.603356 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:57.603360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:57.603361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:57.603601 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:57.603714 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:57.604308 (+ 594us) server_negotiation.cc:299] Negotiation successful
0812 03:07:57.604459 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:07:57.605592 28171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:57.590897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:45331 (local address 127.26.250.131:60177)
0812 03:07:57.591536 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:57.591602 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:57.593104 (+ 1502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:57.593683 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:57.593690 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:57.594075 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:57.594702 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:57.594717 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:57.595595 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:57.595599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:57.596639 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:57.596660 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:57.597288 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:57.597293 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:57.597449 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:57.598155 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:57.598223 (+ 68us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:57.599932 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:57.602407 (+ 2475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:57.602413 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:57.602428 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:57.602760 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:57.603068 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:57.603070 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:57.603071 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:57.603228 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:57.603733 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:57.603739 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:57.604010 (+ 271us) client_negotiation.cc:769] Sending connection context
0812 03:07:57.604181 (+ 171us) client_negotiation.cc:240] Negotiation successful
0812 03:07:57.604575 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":163,"threads_started":1}
I20260812 03:07:57.607156 28169 heartbeater.cc:347] Connected to a master server at 127.26.250.190:45331
I20260812 03:07:57.607477 28169 heartbeater.cc:464] Registering TS with master...
I20260812 03:07:57.608059 28169 heartbeater.cc:511] Master 127.26.250.190:45331 requested a full tablet report, sending...
I20260812 03:07:57.609380 27680 ts_manager.cc:194] Registered new tserver with Master: 7f6b1580daaa49c9a476cea66507e69a (127.26.250.131:42069)
I20260812 03:07:57.610158 27680 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.131@KRBTEST.COM'} at 127.26.250.131:60177
I20260812 03:07:57.618613 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 03:07:57 dist-test-slave-zpfg krb5kdc[27634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504077, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:07:57 dist-test-slave-zpfg krb5kdc[27634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504077, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.250.129@KRBTEST.COM
I20260812 03:07:57.650619 28176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:57.639023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38552 (local address 127.26.250.129:45145)
0812 03:07:57.639457 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:57.639465 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:57.639497 (+ 32us) server_negotiation.cc:407] Connection header received
0812 03:07:57.639548 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:57.639553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:57.639746 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:57.639919 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:57.640675 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:57.641497 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:57.642409 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:57.642812 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:57.642935 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:57.646002 (+ 3067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:57.646013 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:57.646027 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:57.646055 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:57.648303 (+ 2248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:57.648871 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:57.648877 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:57.648879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:57.648982 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:57.649443 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:57.649446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:57.649447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:57.649751 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:57.649933 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:57.650227 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 03:07:57.650465 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":70,"threads_started":1}
W20260812 03:07:57.651811 27792 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38552
Aug 12 03:07:57 dist-test-slave-zpfg krb5kdc[27634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:07:57 dist-test-slave-zpfg krb5kdc[27634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.129@KRBTEST.COM
I20260812 03:07:57.682610 28176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:57.672724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38560 (local address 127.26.250.129:45145)
0812 03:07:57.672916 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:57.672921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:57.673050 (+ 129us) server_negotiation.cc:407] Connection header received
0812 03:07:57.673239 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:57.673247 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:57.673295 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:57.673391 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:57.674267 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:57.674889 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:57.675759 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:57.676012 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:57.676140 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:57.678911 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:57.678916 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:57.678919 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:57.678954 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:57.680577 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:57.681212 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:57.681215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:57.681216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:57.681321 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:57.681583 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:57.681585 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:57.681586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:57.681806 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:57.681932 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:57.682157 (+ 225us) server_negotiation.cc:299] Negotiation successful
0812 03:07:57.682283 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 03:07:57.683995 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 27744
I20260812 03:07:57.693427 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 27889
I20260812 03:07:57.701521 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 28029
I20260812 03:07:57.710901 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 27647
2026-08-12T03:07:57Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2428 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:07:57 dist-test-slave-zpfg krb5kdc[28183](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:07:57 dist-test-slave-zpfg krb5kdc[28183](info): set up 2 sockets
Aug 12 03:07:57 dist-test-slave-zpfg krb5kdc[28183](info): commencing operation
krb5kdc: starting...
W20260812 03:07:58.865996 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:07:58 dist-test-slave-zpfg krb5kdc[28183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:07:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:07:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:07:59.013716 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:41289
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:37315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:41289
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:59.133752 28196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:59.134079 28196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:59.134122 28196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:59.138594 28196 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:07:59.138676 28196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:59.138696 28196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:59.138717 28196 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:07:59.138733 28196 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:07:59.144454 28196 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:37315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:41289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:41289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:59.145664 28196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:59.146725 28196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:59.156041 28202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:59.156098 28201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:59.156296 28196 server_base.cc:1034] running on GCE node
W20260812 03:07:59.156595 28204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:59.157209 28196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:59.158571 28196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:59.159793 28196 hybrid_clock.cc:648] HybridClock initialized: now 1786504079159765 us; error 43 us; skew 500 ppm
Aug 12 03:07:59 dist-test-slave-zpfg krb5kdc[28183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504079, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:59.162984 28196 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:07:59.163985 28196 webserver.cc:460] Webserver started at http://127.26.250.190:32777/ using document root <none> and password file <none>
I20260812 03:07:59.164582 28196 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:59.164646 28196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:59.164875 28196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:59.167093 28196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "bdee3206a7544d0f83ea2aaadab4ee77"
format_stamp: "Formatted at 2026-08-12 03:07:59 on dist-test-slave-zpfg"
server_key: "23beac07ecf64d52ab26ad6125480ac0"
server_key_iv: "d5d94a19b27569f3768ea46c07aabfae"
server_key_version: "encryptionkey@0"
I20260812 03:07:59.167693 28196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "bdee3206a7544d0f83ea2aaadab4ee77"
format_stamp: "Formatted at 2026-08-12 03:07:59 on dist-test-slave-zpfg"
server_key: "23beac07ecf64d52ab26ad6125480ac0"
server_key_iv: "d5d94a19b27569f3768ea46c07aabfae"
server_key_version: "encryptionkey@0"
I20260812 03:07:59.171777 28196 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:07:59.174805 28211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:59.176343 28196 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:07:59.176515 28196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "bdee3206a7544d0f83ea2aaadab4ee77"
format_stamp: "Formatted at 2026-08-12 03:07:59 on dist-test-slave-zpfg"
server_key: "23beac07ecf64d52ab26ad6125480ac0"
server_key_iv: "d5d94a19b27569f3768ea46c07aabfae"
server_key_version: "encryptionkey@0"
I20260812 03:07:59.176659 28196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:59.200146 28196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:59.203833 28196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:59.204062 28196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:59.211905 28196 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:41289
I20260812 03:07:59.211975 28269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:41289 every 8 connection(s)
I20260812 03:07:59.212970 28196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:07:59.216069 28270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:07:59.221922 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 28196
I20260812 03:07:59.222064 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:07:59.222330 27626 external_mini_cluster.cc:1428] Setting key 0994862dc6dc6778810c874b0f6220ea
I20260812 03:07:59.222688 28270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdee3206a7544d0f83ea2aaadab4ee77: Bootstrap starting.
I20260812 03:07:59.225433 28270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdee3206a7544d0f83ea2aaadab4ee77: Neither blocks nor log segments found. Creating new log.
I20260812 03:07:59.226549 28270 log.cc:824] T 00000000000000000000000000000000 P bdee3206a7544d0f83ea2aaadab4ee77: Log is configured to *not* fsync() on all Append() calls
I20260812 03:07:59.229146 28270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdee3206a7544d0f83ea2aaadab4ee77: No bootstrap required, opened a new log
Aug 12 03:07:59 dist-test-slave-zpfg krb5kdc[28183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:07:59.234032 28270 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdee3206a7544d0f83ea2aaadab4ee77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdee3206a7544d0f83ea2aaadab4ee77" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41289 } }
I20260812 03:07:59.234280 28270 raft_consensus.cc:374] T 00000000000000000000000000000000 P bdee3206a7544d0f83ea2aaadab4ee77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:07:59.234396 28270 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdee3206a7544d0f83ea2aaadab4ee77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdee3206a7544d0f83ea2aaadab4ee77, State: Initialized, Role: FOLLOWER
I20260812 03:07:59.234956 28270 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdee3206a7544d0f83ea2aaadab4ee77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdee3206a7544d0f83ea2aaadab4ee77" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41289 } }
I20260812 03:07:59.235096 28270 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdee3206a7544d0f83ea2aaadab4ee77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:07:59.235198 28270 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdee3206a7544d0f83ea2aaadab4ee77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:07:59.235332 28270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdee3206a7544d0f83ea2aaadab4ee77 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:07:59.236603 28270 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdee3206a7544d0f83ea2aaadab4ee77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdee3206a7544d0f83ea2aaadab4ee77" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41289 } }
I20260812 03:07:59.237015 28270 leader_election.cc:302] T 00000000000000000000000000000000 P bdee3206a7544d0f83ea2aaadab4ee77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdee3206a7544d0f83ea2aaadab4ee77; no voters:
I20260812 03:07:59.237416 28270 leader_election.cc:288] T 00000000000000000000000000000000 P bdee3206a7544d0f83ea2aaadab4ee77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:07:59.237584 28275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdee3206a7544d0f83ea2aaadab4ee77 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:07:59.237884 28275 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdee3206a7544d0f83ea2aaadab4ee77 [term 1 LEADER]: Becoming Leader. State: Replica: bdee3206a7544d0f83ea2aaadab4ee77, State: Running, Role: LEADER
I20260812 03:07:59.238281 28275 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdee3206a7544d0f83ea2aaadab4ee77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdee3206a7544d0f83ea2aaadab4ee77" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41289 } }
I20260812 03:07:59.238929 28270 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdee3206a7544d0f83ea2aaadab4ee77 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:07:59.239599 28276 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdee3206a7544d0f83ea2aaadab4ee77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdee3206a7544d0f83ea2aaadab4ee77. Latest consensus state: current_term: 1 leader_uuid: "bdee3206a7544d0f83ea2aaadab4ee77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdee3206a7544d0f83ea2aaadab4ee77" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41289 } } }
I20260812 03:07:59.239724 28276 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdee3206a7544d0f83ea2aaadab4ee77 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:59.239784 28273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:59.223979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58224 (local address 127.26.250.190:41289)
0812 03:07:59.224444 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:59.224461 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:59.224497 (+ 36us) server_negotiation.cc:407] Connection header received
0812 03:07:59.225159 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:59.225177 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:59.225518 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:59.225879 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:59.226940 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:59.228167 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:59.229414 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:59.229893 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:59.230016 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:59.233364 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:59.233370 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:59.233386 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:59.233417 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:59.235533 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:59.236060 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:59.236065 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:59.236070 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:59.236278 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:59.236657 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:59.236659 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:59.236660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:59.237281 (+ 621us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:59.237494 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:59.238262 (+ 768us) server_negotiation.cc:299] Negotiation successful
0812 03:07:59.238635 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":181,"threads_started":1}
I20260812 03:07:59.240440 28277 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdee3206a7544d0f83ea2aaadab4ee77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdee3206a7544d0f83ea2aaadab4ee77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdee3206a7544d0f83ea2aaadab4ee77" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41289 } } }
I20260812 03:07:59.240526 28277 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdee3206a7544d0f83ea2aaadab4ee77 [sys.catalog]: This master's current role is: LEADER
I20260812 03:07:59.241107 28282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:07:59.244230 28282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:07:59.249716 28282 catalog_manager.cc:1347] Generated new cluster ID: d8de26e2f401405e81a08ae97a327fea
I20260812 03:07:59.249800 28282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:07:59.259707 28282 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:07:59.261195 28282 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:07:59.271849 28282 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdee3206a7544d0f83ea2aaadab4ee77: Generated new TSK 0
I20260812 03:07:59.272857 28282 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.250.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
I20260812 03:07:59.309501 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:0
--local_ip_for_outbound_sockets=127.26.250.129
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:41289
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:37315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:59.436352 28297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:59.436694 28297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:59.436779 28297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:59.441408 28297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:59.441524 28297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:59.441658 28297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:07:59.447623 28297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:37315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:41289
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:59.448907 28297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:59.450032 28297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:59.460963 28304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:59.460963 28307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:59.461340 28297 server_base.cc:1034] running on GCE node
W20260812 03:07:59.461428 28305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:59.461884 28297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:59.462659 28297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:59.463877 28297 hybrid_clock.cc:648] HybridClock initialized: now 1786504079463860 us; error 29 us; skew 500 ppm
Aug 12 03:07:59 dist-test-slave-zpfg krb5kdc[28183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504079, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:59.467165 28297 init.cc:377] Logged in from keytab as kudu/127.26.250.129@KRBTEST.COM (short username kudu)
I20260812 03:07:59.468133 28297 webserver.cc:460] Webserver started at http://127.26.250.129:39673/ using document root <none> and password file <none>
I20260812 03:07:59.468740 28297 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:59.468837 28297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:59.469075 28297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:59.471323 28297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-0/data/instance:
uuid: "dd434da39a154335ac952748eb1d1fbd"
format_stamp: "Formatted at 2026-08-12 03:07:59 on dist-test-slave-zpfg"
server_key: "1642f7e537fa374a280f2b0a5500942c"
server_key_iv: "57cab1b6a2dff91c7239677067ecd75d"
server_key_version: "encryptionkey@0"
I20260812 03:07:59.471915 28297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance:
uuid: "dd434da39a154335ac952748eb1d1fbd"
format_stamp: "Formatted at 2026-08-12 03:07:59 on dist-test-slave-zpfg"
server_key: "1642f7e537fa374a280f2b0a5500942c"
server_key_iv: "57cab1b6a2dff91c7239677067ecd75d"
server_key_version: "encryptionkey@0"
I20260812 03:07:59.476027 28297 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:07:59.479038 28315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:59.480739 28297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:07:59.480918 28297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "dd434da39a154335ac952748eb1d1fbd"
format_stamp: "Formatted at 2026-08-12 03:07:59 on dist-test-slave-zpfg"
server_key: "1642f7e537fa374a280f2b0a5500942c"
server_key_iv: "57cab1b6a2dff91c7239677067ecd75d"
server_key_version: "encryptionkey@0"
I20260812 03:07:59.481058 28297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:59.499744 28297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:59.503268 28297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:59.503485 28297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:59.504127 28297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:07:59.504998 28297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:07:59.505072 28297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:59.505143 28297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:07:59.505195 28297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:59.514997 28297 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:33883
I20260812 03:07:59.515021 28433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:33883 every 8 connection(s)
I20260812 03:07:59.515983 28297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:07:59.518075 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 28297
I20260812 03:07:59.518289 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance
I20260812 03:07:59.518615 27626 external_mini_cluster.cc:1428] Setting key 3c68ddcf1dd01d60022501207f2abe06
Aug 12 03:07:59 dist-test-slave-zpfg krb5kdc[28183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504079, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.250.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.130@KRBTEST.COM" created.
I20260812 03:07:59.535089 28273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:59.518061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:47927 (local address 127.26.250.190:41289)
0812 03:07:59.518213 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:59.518218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:59.519422 (+ 1204us) server_negotiation.cc:407] Connection header received
0812 03:07:59.520659 (+ 1237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:59.520662 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:59.520763 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:59.520851 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:59.522926 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:59.523971 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:59.526104 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:59.526450 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:59.526554 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:59.530040 (+ 3486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:59.530046 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:59.530049 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:59.530085 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:59.531964 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:59.532712 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:59.532718 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:59.532720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:59.532829 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:59.533434 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:59.533437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:59.533438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:59.533751 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:59.533862 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:59.534632 (+ 770us) server_negotiation.cc:299] Negotiation successful
0812 03:07:59.534819 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 03:07:59.536000 28436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:59.518545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:41289 (local address 127.26.250.129:47927)
0812 03:07:59.519189 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:59.519239 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:59.520384 (+ 1145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:59.521118 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:59.521131 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:59.521791 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:59.522664 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:59.522692 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:59.524156 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:59.524159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:59.525923 (+ 1764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:59.525941 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:59.526651 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:59.526658 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:59.526937 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:59.527736 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:59.527765 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:59.529804 (+ 2039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:59.532122 (+ 2318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:59.532129 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:59.532148 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:59.532534 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:59.532967 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:59.532971 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:59.532972 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:59.533148 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:59.533940 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:59.533946 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:59.534266 (+ 320us) client_negotiation.cc:769] Sending connection context
0812 03:07:59.534524 (+ 258us) client_negotiation.cc:240] Negotiation successful
0812 03:07:59.534958 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":195,"threads_started":1}
I20260812 03:07:59.537658 28434 heartbeater.cc:347] Connected to a master server at 127.26.250.190:41289
I20260812 03:07:59.537976 28434 heartbeater.cc:464] Registering TS with master...
I20260812 03:07:59.538652 28434 heartbeater.cc:511] Master 127.26.250.190:41289 requested a full tablet report, sending...
I20260812 03:07:59.540769 28229 ts_manager.cc:194] Registered new tserver with Master: dd434da39a154335ac952748eb1d1fbd (127.26.250.129:33883)
I20260812 03:07:59.542302 28229 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.129@KRBTEST.COM'} at 127.26.250.129:47927
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
I20260812 03:07:59.554601 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:0
--local_ip_for_outbound_sockets=127.26.250.130
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:41289
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:37315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:59.681519 28440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:59.681867 28440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:59.681957 28440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:59.686553 28440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:59.686676 28440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:59.686803 28440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:07:59.692559 28440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:37315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:41289
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:59.693778 28440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:59.694917 28440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:59.705924 28448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:59.705925 28445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:59.705924 28446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:59.706023 28440 server_base.cc:1034] running on GCE node
I20260812 03:07:59.706759 28440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:59.707479 28440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:59.708714 28440 hybrid_clock.cc:648] HybridClock initialized: now 1786504079708698 us; error 40 us; skew 500 ppm
Aug 12 03:07:59 dist-test-slave-zpfg krb5kdc[28183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504079, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:59.712329 28440 init.cc:377] Logged in from keytab as kudu/127.26.250.130@KRBTEST.COM (short username kudu)
I20260812 03:07:59.713382 28440 webserver.cc:460] Webserver started at http://127.26.250.130:45409/ using document root <none> and password file <none>
I20260812 03:07:59.714056 28440 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:59.714165 28440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:59.714505 28440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:59.716948 28440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-1/data/instance:
uuid: "0e7c358e873e4d91870d10a3afead870"
format_stamp: "Formatted at 2026-08-12 03:07:59 on dist-test-slave-zpfg"
server_key: "c7442aa92fafea05ab7e6f166a175183"
server_key_iv: "a87026a43b4f87bcc6f52961ba52e744"
server_key_version: "encryptionkey@0"
I20260812 03:07:59.717662 28440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance:
uuid: "0e7c358e873e4d91870d10a3afead870"
format_stamp: "Formatted at 2026-08-12 03:07:59 on dist-test-slave-zpfg"
server_key: "c7442aa92fafea05ab7e6f166a175183"
server_key_iv: "a87026a43b4f87bcc6f52961ba52e744"
server_key_version: "encryptionkey@0"
I20260812 03:07:59.722533 28440 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260812 03:07:59.725539 28456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:59.727110 28440 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:07:59.727295 28440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "0e7c358e873e4d91870d10a3afead870"
format_stamp: "Formatted at 2026-08-12 03:07:59 on dist-test-slave-zpfg"
server_key: "c7442aa92fafea05ab7e6f166a175183"
server_key_iv: "a87026a43b4f87bcc6f52961ba52e744"
server_key_version: "encryptionkey@0"
I20260812 03:07:59.727444 28440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:59.744415 28440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:59.748389 28440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:59.748646 28440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:59.749366 28440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:07:59.750429 28440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:07:59.750507 28440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:59.750587 28440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:07:59.750643 28440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:59.761243 28440 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:46325
I20260812 03:07:59.761269 28572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:46325 every 8 connection(s)
I20260812 03:07:59.762440 28440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:07:59.763273 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 28440
I20260812 03:07:59.763396 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance
I20260812 03:07:59.763720 27626 external_mini_cluster.cc:1428] Setting key ed6e00830585c02f8154453c403d7ba9
Aug 12 03:07:59 dist-test-slave-zpfg krb5kdc[28183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504079, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.250.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.131@KRBTEST.COM" created.
I20260812 03:07:59.779251 28273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:59.764683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:53821 (local address 127.26.250.190:41289)
0812 03:07:59.764820 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 03:07:59.764824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:07:59.766284 (+ 1460us) server_negotiation.cc:407] Connection header received
0812 03:07:59.767486 (+ 1202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:07:59.767490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:07:59.767544 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:07:59.767630 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:07:59.768947 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:59.769496 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:59.770789 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:07:59.771024 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:07:59.771150 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:59.774540 (+ 3390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:07:59.774544 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:07:59.774546 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:07:59.774575 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:07:59.776204 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:59.776882 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:59.776885 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:59.776886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:59.776991 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:07:59.777601 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:07:59.777603 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:07:59.777605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:07:59.777865 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:07:59.777950 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 03:07:59.778857 (+ 907us) server_negotiation.cc:299] Negotiation successful
0812 03:07:59.779025 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 03:07:59.780179 28575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:07:59.765459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:41289 (local address 127.26.250.130:53821)
0812 03:07:59.766112 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0812 03:07:59.766159 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:07:59.767170 (+ 1011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:07:59.767829 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:07:59.767836 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:07:59.768247 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:07:59.768794 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:59.768809 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:59.769639 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:59.769642 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:59.770661 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:07:59.770671 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:07:59.771213 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:07:59.771221 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:07:59.771389 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:07:59.772208 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:07:59.772232 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:07:59.774295 (+ 2063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:07:59.776372 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:59.776378 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:59.776394 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:59.776715 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:59.777095 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:07:59.777098 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:07:59.777100 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:07:59.777472 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:07:59.777998 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:07:59.778007 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:07:59.778545 (+ 538us) client_negotiation.cc:769] Sending connection context
0812 03:07:59.778739 (+ 194us) client_negotiation.cc:240] Negotiation successful
0812 03:07:59.779061 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":182,"threads_started":1}
I20260812 03:07:59.781780 28573 heartbeater.cc:347] Connected to a master server at 127.26.250.190:41289
I20260812 03:07:59.782159 28573 heartbeater.cc:464] Registering TS with master...
I20260812 03:07:59.782928 28573 heartbeater.cc:511] Master 127.26.250.190:41289 requested a full tablet report, sending...
I20260812 03:07:59.784286 28229 ts_manager.cc:194] Registered new tserver with Master: 0e7c358e873e4d91870d10a3afead870 (127.26.250.130:46325)
I20260812 03:07:59.785264 28229 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.130@KRBTEST.COM'} at 127.26.250.130:53821
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
I20260812 03:07:59.798676 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:0
--local_ip_for_outbound_sockets=127.26.250.131
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:41289
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:37315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:07:59.919298 28578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:07:59.919615 28578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:07:59.919661 28578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:07:59.924708 28578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:07:59.924799 28578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:07:59.924907 28578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:07:59.930964 28578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:37315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:41289
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:07:59.932241 28578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:07:59.933403 28578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:07:59.944059 28583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:59.944059 28584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:07:59.944096 28586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:07:59.944536 28578 server_base.cc:1034] running on GCE node
I20260812 03:07:59.945001 28578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:07:59.945789 28578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:07:59.947013 28578 hybrid_clock.cc:648] HybridClock initialized: now 1786504079946987 us; error 39 us; skew 500 ppm
Aug 12 03:07:59 dist-test-slave-zpfg krb5kdc[28183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504079, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:07:59.950212 28578 init.cc:377] Logged in from keytab as kudu/127.26.250.131@KRBTEST.COM (short username kudu)
I20260812 03:07:59.951453 28578 webserver.cc:460] Webserver started at http://127.26.250.131:39253/ using document root <none> and password file <none>
I20260812 03:07:59.952030 28578 fs_manager.cc:359] Metadata directory not provided
I20260812 03:07:59.952093 28578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:07:59.952275 28578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:07:59.954504 28578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-2/data/instance:
uuid: "715dc1bd8c2f4e85ad878fe61b6baf0a"
format_stamp: "Formatted at 2026-08-12 03:07:59 on dist-test-slave-zpfg"
server_key: "c156ad2db56c2c013b5ae707798d6ee6"
server_key_iv: "be51627ead0d2f32cac8f1c43061eedc"
server_key_version: "encryptionkey@0"
I20260812 03:07:59.955075 28578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance:
uuid: "715dc1bd8c2f4e85ad878fe61b6baf0a"
format_stamp: "Formatted at 2026-08-12 03:07:59 on dist-test-slave-zpfg"
server_key: "c156ad2db56c2c013b5ae707798d6ee6"
server_key_iv: "be51627ead0d2f32cac8f1c43061eedc"
server_key_version: "encryptionkey@0"
I20260812 03:07:59.959108 28578 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:07:59.961686 28593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:59.963259 28578 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:07:59.963370 28578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "715dc1bd8c2f4e85ad878fe61b6baf0a"
format_stamp: "Formatted at 2026-08-12 03:07:59 on dist-test-slave-zpfg"
server_key: "c156ad2db56c2c013b5ae707798d6ee6"
server_key_iv: "be51627ead0d2f32cac8f1c43061eedc"
server_key_version: "encryptionkey@0"
I20260812 03:07:59.963465 28578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:07:59.990293 28578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:07:59.994097 28578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:07:59.994282 28578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:07:59.994971 28578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:07:59.995838 28578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:07:59.995888 28578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:07:59.995929 28578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:07:59.995945 28578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:00.007817 28578 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:44965
I20260812 03:08:00.007925 28713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:44965 every 8 connection(s)
I20260812 03:08:00.008926 28578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:08:00.017922 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 28578
I20260812 03:08:00.018121 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance
I20260812 03:08:00.018592 27626 external_mini_cluster.cc:1428] Setting key eb7c87079f46062b1170cd2d53a744cc
Aug 12 03:08:00 dist-test-slave-zpfg krb5kdc[28183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504079, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:00.025259 28273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:00.011301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:39545 (local address 127.26.250.190:41289)
0812 03:08:00.011457 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:00.011461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:00.012347 (+ 886us) server_negotiation.cc:407] Connection header received
0812 03:08:00.013345 (+ 998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:00.013348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:00.013413 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:00.013525 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:00.014972 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.015541 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.016807 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.017091 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.017226 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:00.020617 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:00.020621 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:00.020623 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:00.020656 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:00.022712 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:00.023405 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:00.023408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:00.023409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:00.023496 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:00.023906 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:00.023908 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:00.023910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:00.024210 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:00.024319 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:00.024956 (+ 637us) server_negotiation.cc:299] Negotiation successful
0812 03:08:00.025108 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 03:08:00.026320 28716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:00.011593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:41289 (local address 127.26.250.131:39545)
0812 03:08:00.012145 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:00.012213 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:00.013114 (+ 901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:00.013766 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:00.013774 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:00.014174 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:00.014810 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:00.014826 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.015701 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.015704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:00.016662 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:00.016673 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.017331 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.017335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:00.017514 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:00.018263 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:00.018282 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:00.020434 (+ 2152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:00.022860 (+ 2426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:00.022866 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:00.022882 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:00.023193 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:00.023594 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:00.023597 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:00.023598 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:00.023764 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:00.024356 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:00.024362 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:00.024678 (+ 316us) client_negotiation.cc:769] Sending connection context
0812 03:08:00.024873 (+ 195us) client_negotiation.cc:240] Negotiation successful
0812 03:08:00.025175 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":149,"threads_started":1}
I20260812 03:08:00.027869 28714 heartbeater.cc:347] Connected to a master server at 127.26.250.190:41289
I20260812 03:08:00.028177 28714 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:00.028822 28714 heartbeater.cc:511] Master 127.26.250.190:41289 requested a full tablet report, sending...
I20260812 03:08:00.030098 28229 ts_manager.cc:194] Registered new tserver with Master: 715dc1bd8c2f4e85ad878fe61b6baf0a (127.26.250.131:44965)
I20260812 03:08:00.030788 28229 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.131@KRBTEST.COM'} at 127.26.250.131:39545
I20260812 03:08:00.034235 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:08:00.047634 28273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:00.038215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58238 (local address 127.26.250.190:41289)
0812 03:08:00.038429 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:00.038433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:00.038606 (+ 173us) server_negotiation.cc:407] Connection header received
0812 03:08:00.038786 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:00.038789 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:00.038841 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:00.038916 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:00.040109 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.040903 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.041823 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.042070 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.042177 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:00.043990 (+ 1813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:00.043993 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:00.043995 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:00.044024 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:00.045566 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:00.046023 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:00.046026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:00.046027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:00.046114 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:00.046636 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:00.046639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:00.046640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:00.046895 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:00.047015 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:00.047354 (+ 339us) server_negotiation.cc:299] Negotiation successful
0812 03:08:00.047493 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 03:08:00.053583 28229 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58238:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:08:00.057156 28229 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:08:00.074163 28726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:00.067584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:44965 (local address 127.0.0.1:60070)
0812 03:08:00.068345 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:00.068402 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:00.068690 (+ 288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:00.069655 (+ 965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:00.069659 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:00.069680 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:00.069820 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:00.069827 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.071339 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.071342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:00.072601 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:00.072611 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.073506 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.073510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:00.073648 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:00.073707 (+ 59us) client_negotiation.cc:769] Sending connection context
0812 03:08:00.073835 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 03:08:00.073972 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":1,"thread_start_us":110,"threads_started":1}
I20260812 03:08:00.074156 28729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:00.067938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60070 (local address 127.26.250.131:44965)
0812 03:08:00.068367 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:00.068381 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:00.068683 (+ 302us) server_negotiation.cc:407] Connection header received
0812 03:08:00.069300 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:00.069305 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:00.069490 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:00.069647 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:00.069956 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.071148 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.072737 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.073378 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.073508 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:00.073650 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:00.073884 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 03:08:00.073990 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":151,"threads_started":1}
I20260812 03:08:00.075089 28727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:00.067584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:46325 (local address 127.0.0.1:60272)
0812 03:08:00.068609 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:00.068625 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:00.068802 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:00.069477 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:00.069482 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:00.069505 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:00.069676 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:00.069685 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.071469 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.071473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:00.073219 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:00.073231 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.074620 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.074624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:00.074735 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:00.074759 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:08:00.074838 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 03:08:00.074924 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":937,"mutex_wait_us":234,"thread_start_us":110,"threads_started":1}
I20260812 03:08:00.075248 28730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:00.068469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:33883 (local address 127.0.0.1:60898)
0812 03:08:00.069018 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:00.069032 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:00.069130 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:00.070288 (+ 1158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:00.070291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:00.070310 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:00.070443 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:00.070451 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.071636 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.071640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:00.073219 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:00.073231 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.074702 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.074707 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:00.074851 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:00.074879 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 03:08:00.074955 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 03:08:00.075036 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":203,"thread_start_us":101,"threads_started":1}
I20260812 03:08:00.075431 28731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:00.068938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60898 (local address 127.26.250.129:33883)
0812 03:08:00.069877 (+ 939us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:00.069888 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:00.069915 (+ 27us) server_negotiation.cc:407] Connection header received
0812 03:08:00.070005 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:00.070009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:00.070152 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:00.070314 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:00.070576 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.071495 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.073392 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.074287 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.074477 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:00.074621 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:00.075163 (+ 542us) server_negotiation.cc:299] Negotiation successful
0812 03:08:00.075270 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":772,"thread_start_us":119,"threads_started":1}
I20260812 03:08:00.075728 28728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:00.067774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60272 (local address 127.26.250.130:46325)
0812 03:08:00.068594 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:00.068610 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:00.068917 (+ 307us) server_negotiation.cc:407] Connection header received
0812 03:08:00.069015 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:00.069022 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:00.069287 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:00.069453 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:00.069835 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.071156 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.073371 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.074260 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.074425 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:00.074548 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:00.075457 (+ 909us) server_negotiation.cc:299] Negotiation successful
0812 03:08:00.075561 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":130,"threads_started":1}
I20260812 03:08:00.078117 28364 tablet_service.cc:1467] Processing CreateTablet for tablet 79f2720ea4764a6f94e6b494a16ef628 (DEFAULT_TABLE table=test-table [id=6ebcc8f75fc14b6c809920c9327298cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:00.078019 28646 tablet_service.cc:1467] Processing CreateTablet for tablet 79f2720ea4764a6f94e6b494a16ef628 (DEFAULT_TABLE table=test-table [id=6ebcc8f75fc14b6c809920c9327298cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:00.079229 28364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79f2720ea4764a6f94e6b494a16ef628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:00.079286 28646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79f2720ea4764a6f94e6b494a16ef628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:00.080718 28504 tablet_service.cc:1467] Processing CreateTablet for tablet 79f2720ea4764a6f94e6b494a16ef628 (DEFAULT_TABLE table=test-table [id=6ebcc8f75fc14b6c809920c9327298cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:00.081983 28504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79f2720ea4764a6f94e6b494a16ef628. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:00.086594 28732 tablet_bootstrap.cc:492] T 79f2720ea4764a6f94e6b494a16ef628 P dd434da39a154335ac952748eb1d1fbd: Bootstrap starting.
I20260812 03:08:00.088238 28733 tablet_bootstrap.cc:492] T 79f2720ea4764a6f94e6b494a16ef628 P 715dc1bd8c2f4e85ad878fe61b6baf0a: Bootstrap starting.
I20260812 03:08:00.089182 28732 tablet_bootstrap.cc:654] T 79f2720ea4764a6f94e6b494a16ef628 P dd434da39a154335ac952748eb1d1fbd: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:00.089797 28734 tablet_bootstrap.cc:492] T 79f2720ea4764a6f94e6b494a16ef628 P 0e7c358e873e4d91870d10a3afead870: Bootstrap starting.
I20260812 03:08:00.090106 28732 log.cc:824] T 79f2720ea4764a6f94e6b494a16ef628 P dd434da39a154335ac952748eb1d1fbd: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:00.090849 28733 tablet_bootstrap.cc:654] T 79f2720ea4764a6f94e6b494a16ef628 P 715dc1bd8c2f4e85ad878fe61b6baf0a: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:00.092075 28733 log.cc:824] T 79f2720ea4764a6f94e6b494a16ef628 P 715dc1bd8c2f4e85ad878fe61b6baf0a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:00.092394 28734 tablet_bootstrap.cc:654] T 79f2720ea4764a6f94e6b494a16ef628 P 0e7c358e873e4d91870d10a3afead870: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:00.093195 28734 log.cc:824] T 79f2720ea4764a6f94e6b494a16ef628 P 0e7c358e873e4d91870d10a3afead870: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:00.094286 28732 tablet_bootstrap.cc:492] T 79f2720ea4764a6f94e6b494a16ef628 P dd434da39a154335ac952748eb1d1fbd: No bootstrap required, opened a new log
I20260812 03:08:00.094310 28733 tablet_bootstrap.cc:492] T 79f2720ea4764a6f94e6b494a16ef628 P 715dc1bd8c2f4e85ad878fe61b6baf0a: No bootstrap required, opened a new log
I20260812 03:08:00.094538 28733 ts_tablet_manager.cc:1397] T 79f2720ea4764a6f94e6b494a16ef628 P 715dc1bd8c2f4e85ad878fe61b6baf0a: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260812 03:08:00.094551 28732 ts_tablet_manager.cc:1397] T 79f2720ea4764a6f94e6b494a16ef628 P dd434da39a154335ac952748eb1d1fbd: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 03:08:00.095322 28734 tablet_bootstrap.cc:492] T 79f2720ea4764a6f94e6b494a16ef628 P 0e7c358e873e4d91870d10a3afead870: No bootstrap required, opened a new log
I20260812 03:08:00.095516 28734 ts_tablet_manager.cc:1397] T 79f2720ea4764a6f94e6b494a16ef628 P 0e7c358e873e4d91870d10a3afead870: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:08:00.098562 28732 raft_consensus.cc:348] T 79f2720ea4764a6f94e6b494a16ef628 P dd434da39a154335ac952748eb1d1fbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7c358e873e4d91870d10a3afead870" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46325 } } peers { permanent_uuid: "715dc1bd8c2f4e85ad878fe61b6baf0a" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 44965 } } peers { permanent_uuid: "dd434da39a154335ac952748eb1d1fbd" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 33883 } }
I20260812 03:08:00.098749 28733 raft_consensus.cc:348] T 79f2720ea4764a6f94e6b494a16ef628 P 715dc1bd8c2f4e85ad878fe61b6baf0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7c358e873e4d91870d10a3afead870" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46325 } } peers { permanent_uuid: "715dc1bd8c2f4e85ad878fe61b6baf0a" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 44965 } } peers { permanent_uuid: "dd434da39a154335ac952748eb1d1fbd" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 33883 } }
I20260812 03:08:00.098968 28732 raft_consensus.cc:374] T 79f2720ea4764a6f94e6b494a16ef628 P dd434da39a154335ac952748eb1d1fbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:00.099045 28732 raft_consensus.cc:729] T 79f2720ea4764a6f94e6b494a16ef628 P dd434da39a154335ac952748eb1d1fbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd434da39a154335ac952748eb1d1fbd, State: Initialized, Role: FOLLOWER
I20260812 03:08:00.099049 28733 raft_consensus.cc:374] T 79f2720ea4764a6f94e6b494a16ef628 P 715dc1bd8c2f4e85ad878fe61b6baf0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:00.099102 28733 raft_consensus.cc:729] T 79f2720ea4764a6f94e6b494a16ef628 P 715dc1bd8c2f4e85ad878fe61b6baf0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 715dc1bd8c2f4e85ad878fe61b6baf0a, State: Initialized, Role: FOLLOWER
I20260812 03:08:00.099466 28734 raft_consensus.cc:348] T 79f2720ea4764a6f94e6b494a16ef628 P 0e7c358e873e4d91870d10a3afead870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7c358e873e4d91870d10a3afead870" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46325 } } peers { permanent_uuid: "715dc1bd8c2f4e85ad878fe61b6baf0a" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 44965 } } peers { permanent_uuid: "dd434da39a154335ac952748eb1d1fbd" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 33883 } }
I20260812 03:08:00.099629 28733 consensus_queue.cc:260] T 79f2720ea4764a6f94e6b494a16ef628 P 715dc1bd8c2f4e85ad878fe61b6baf0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7c358e873e4d91870d10a3afead870" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46325 } } peers { permanent_uuid: "715dc1bd8c2f4e85ad878fe61b6baf0a" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 44965 } } peers { permanent_uuid: "dd434da39a154335ac952748eb1d1fbd" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 33883 } }
I20260812 03:08:00.099627 28732 consensus_queue.cc:260] T 79f2720ea4764a6f94e6b494a16ef628 P dd434da39a154335ac952748eb1d1fbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7c358e873e4d91870d10a3afead870" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46325 } } peers { permanent_uuid: "715dc1bd8c2f4e85ad878fe61b6baf0a" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 44965 } } peers { permanent_uuid: "dd434da39a154335ac952748eb1d1fbd" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 33883 } }
I20260812 03:08:00.099746 28734 raft_consensus.cc:374] T 79f2720ea4764a6f94e6b494a16ef628 P 0e7c358e873e4d91870d10a3afead870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:00.099802 28734 raft_consensus.cc:729] T 79f2720ea4764a6f94e6b494a16ef628 P 0e7c358e873e4d91870d10a3afead870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e7c358e873e4d91870d10a3afead870, State: Initialized, Role: FOLLOWER
I20260812 03:08:00.100196 28734 consensus_queue.cc:260] T 79f2720ea4764a6f94e6b494a16ef628 P 0e7c358e873e4d91870d10a3afead870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7c358e873e4d91870d10a3afead870" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46325 } } peers { permanent_uuid: "715dc1bd8c2f4e85ad878fe61b6baf0a" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 44965 } } peers { permanent_uuid: "dd434da39a154335ac952748eb1d1fbd" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 33883 } }
I20260812 03:08:00.100543 28434 heartbeater.cc:503] Master 127.26.250.190:41289 was elected leader, sending a full tablet report...
I20260812 03:08:00.100771 28733 ts_tablet_manager.cc:1428] T 79f2720ea4764a6f94e6b494a16ef628 P 715dc1bd8c2f4e85ad878fe61b6baf0a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 03:08:00.100867 28732 ts_tablet_manager.cc:1428] T 79f2720ea4764a6f94e6b494a16ef628 P dd434da39a154335ac952748eb1d1fbd: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 03:08:00.101050 28734 ts_tablet_manager.cc:1428] T 79f2720ea4764a6f94e6b494a16ef628 P 0e7c358e873e4d91870d10a3afead870: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 03:08:00.101148 28714 heartbeater.cc:503] Master 127.26.250.190:41289 was elected leader, sending a full tablet report...
I20260812 03:08:00.101384 28573 heartbeater.cc:503] Master 127.26.250.190:41289 was elected leader, sending a full tablet report...
W20260812 03:08:00.260042 28715 tablet.cc:2369] T 79f2720ea4764a6f94e6b494a16ef628 P 715dc1bd8c2f4e85ad878fe61b6baf0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:08:00.264074 28574 tablet.cc:2369] T 79f2720ea4764a6f94e6b494a16ef628 P 0e7c358e873e4d91870d10a3afead870: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:08:00.267473 28435 tablet.cc:2369] T 79f2720ea4764a6f94e6b494a16ef628 P dd434da39a154335ac952748eb1d1fbd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:08:00.308633 28739 raft_consensus.cc:482] T 79f2720ea4764a6f94e6b494a16ef628 P 715dc1bd8c2f4e85ad878fe61b6baf0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:08:00.308934 28739 raft_consensus.cc:504] T 79f2720ea4764a6f94e6b494a16ef628 P 715dc1bd8c2f4e85ad878fe61b6baf0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7c358e873e4d91870d10a3afead870" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46325 } } peers { permanent_uuid: "715dc1bd8c2f4e85ad878fe61b6baf0a" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 44965 } } peers { permanent_uuid: "dd434da39a154335ac952748eb1d1fbd" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 33883 } }
I20260812 03:08:00.310312 28739 leader_election.cc:288] T 79f2720ea4764a6f94e6b494a16ef628 P 715dc1bd8c2f4e85ad878fe61b6baf0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e7c358e873e4d91870d10a3afead870 (127.26.250.130:46325), dd434da39a154335ac952748eb1d1fbd (127.26.250.129:33883)
I20260812 03:08:00.315214 28716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:00.310618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:46325 (local address 127.26.250.131:49915)
0812 03:08:00.310831 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:00.310850 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:00.310983 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:00.311500 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:00.311503 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:00.311527 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:00.311664 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:00.311672 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.312680 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.312683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:00.313705 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:00.313716 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.314646 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.314650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:00.314814 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:00.314845 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 03:08:00.314946 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 03:08:00.315040 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":55}
I20260812 03:08:00.315366 28728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:00.310715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:49915 (local address 127.26.250.130:46325)
0812 03:08:00.310887 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:00.310894 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:00.310913 (+ 19us) server_negotiation.cc:407] Connection header received
0812 03:08:00.311169 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:00.311173 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:00.311235 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:00.311315 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:00.311843 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.312537 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.313863 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.314515 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.314627 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:00.314674 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:00.315022 (+ 348us) server_negotiation.cc:299] Negotiation successful
0812 03:08:00.315108 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 03:08:00.315490 28742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:00.310981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:33883 (local address 127.26.250.131:41907)
0812 03:08:00.311381 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:00.311395 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:00.311508 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:00.311829 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:00.311833 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:00.311856 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:00.311998 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:00.312005 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.312902 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.312905 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:00.313952 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:00.313964 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.315047 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.315050 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:00.315155 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:00.315179 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:08:00.315270 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 03:08:00.315339 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":21,"thread_start_us":171,"threads_started":1}
I20260812 03:08:00.315542 28731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:00.311186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:41907 (local address 127.26.250.129:33883)
0812 03:08:00.311339 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:00.311344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:00.311495 (+ 151us) server_negotiation.cc:407] Connection header received
0812 03:08:00.311589 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:00.311592 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:00.311654 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:00.311749 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:00.312174 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.312770 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.314149 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.314909 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.315029 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:00.315085 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:00.315307 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 03:08:00.315393 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 03:08:00.316210 28386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79f2720ea4764a6f94e6b494a16ef628" candidate_uuid: "715dc1bd8c2f4e85ad878fe61b6baf0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd434da39a154335ac952748eb1d1fbd" is_pre_election: true
I20260812 03:08:00.316320 28525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79f2720ea4764a6f94e6b494a16ef628" candidate_uuid: "715dc1bd8c2f4e85ad878fe61b6baf0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e7c358e873e4d91870d10a3afead870" is_pre_election: true
I20260812 03:08:00.316604 28386 raft_consensus.cc:2454] T 79f2720ea4764a6f94e6b494a16ef628 P dd434da39a154335ac952748eb1d1fbd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 715dc1bd8c2f4e85ad878fe61b6baf0a in term 0.
I20260812 03:08:00.316677 28525 raft_consensus.cc:2454] T 79f2720ea4764a6f94e6b494a16ef628 P 0e7c358e873e4d91870d10a3afead870 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 715dc1bd8c2f4e85ad878fe61b6baf0a in term 0.
I20260812 03:08:00.317196 28594 leader_election.cc:302] T 79f2720ea4764a6f94e6b494a16ef628 P 715dc1bd8c2f4e85ad878fe61b6baf0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 715dc1bd8c2f4e85ad878fe61b6baf0a, dd434da39a154335ac952748eb1d1fbd; no voters:
I20260812 03:08:00.317512 28739 raft_consensus.cc:2781] T 79f2720ea4764a6f94e6b494a16ef628 P 715dc1bd8c2f4e85ad878fe61b6baf0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:08:00.317595 28739 raft_consensus.cc:482] T 79f2720ea4764a6f94e6b494a16ef628 P 715dc1bd8c2f4e85ad878fe61b6baf0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:08:00.317626 28739 raft_consensus.cc:3037] T 79f2720ea4764a6f94e6b494a16ef628 P 715dc1bd8c2f4e85ad878fe61b6baf0a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:00.318838 28739 raft_consensus.cc:504] T 79f2720ea4764a6f94e6b494a16ef628 P 715dc1bd8c2f4e85ad878fe61b6baf0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7c358e873e4d91870d10a3afead870" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46325 } } peers { permanent_uuid: "715dc1bd8c2f4e85ad878fe61b6baf0a" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 44965 } } peers { permanent_uuid: "dd434da39a154335ac952748eb1d1fbd" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 33883 } }
I20260812 03:08:00.319175 28739 leader_election.cc:288] T 79f2720ea4764a6f94e6b494a16ef628 P 715dc1bd8c2f4e85ad878fe61b6baf0a [CANDIDATE]: Term 1 election: Requested vote from peers 0e7c358e873e4d91870d10a3afead870 (127.26.250.130:46325), dd434da39a154335ac952748eb1d1fbd (127.26.250.129:33883)
I20260812 03:08:00.319711 28525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79f2720ea4764a6f94e6b494a16ef628" candidate_uuid: "715dc1bd8c2f4e85ad878fe61b6baf0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e7c358e873e4d91870d10a3afead870"
I20260812 03:08:00.319762 28386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79f2720ea4764a6f94e6b494a16ef628" candidate_uuid: "715dc1bd8c2f4e85ad878fe61b6baf0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd434da39a154335ac952748eb1d1fbd"
I20260812 03:08:00.319868 28525 raft_consensus.cc:3037] T 79f2720ea4764a6f94e6b494a16ef628 P 0e7c358e873e4d91870d10a3afead870 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:00.319885 28386 raft_consensus.cc:3037] T 79f2720ea4764a6f94e6b494a16ef628 P dd434da39a154335ac952748eb1d1fbd [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:00.321390 28525 raft_consensus.cc:2454] T 79f2720ea4764a6f94e6b494a16ef628 P 0e7c358e873e4d91870d10a3afead870 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 715dc1bd8c2f4e85ad878fe61b6baf0a in term 1.
I20260812 03:08:00.321390 28386 raft_consensus.cc:2454] T 79f2720ea4764a6f94e6b494a16ef628 P dd434da39a154335ac952748eb1d1fbd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 715dc1bd8c2f4e85ad878fe61b6baf0a in term 1.
I20260812 03:08:00.321768 28594 leader_election.cc:302] T 79f2720ea4764a6f94e6b494a16ef628 P 715dc1bd8c2f4e85ad878fe61b6baf0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e7c358e873e4d91870d10a3afead870, 715dc1bd8c2f4e85ad878fe61b6baf0a; no voters:
I20260812 03:08:00.321983 28739 raft_consensus.cc:2781] T 79f2720ea4764a6f94e6b494a16ef628 P 715dc1bd8c2f4e85ad878fe61b6baf0a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:00.322292 28739 raft_consensus.cc:686] T 79f2720ea4764a6f94e6b494a16ef628 P 715dc1bd8c2f4e85ad878fe61b6baf0a [term 1 LEADER]: Becoming Leader. State: Replica: 715dc1bd8c2f4e85ad878fe61b6baf0a, State: Running, Role: LEADER
I20260812 03:08:00.322736 28739 consensus_queue.cc:237] T 79f2720ea4764a6f94e6b494a16ef628 P 715dc1bd8c2f4e85ad878fe61b6baf0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7c358e873e4d91870d10a3afead870" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46325 } } peers { permanent_uuid: "715dc1bd8c2f4e85ad878fe61b6baf0a" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 44965 } } peers { permanent_uuid: "dd434da39a154335ac952748eb1d1fbd" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 33883 } }
I20260812 03:08:00.326136 28229 catalog_manager.cc:5495] T 79f2720ea4764a6f94e6b494a16ef628 P 715dc1bd8c2f4e85ad878fe61b6baf0a reported cstate change: term changed from 0 to 1, leader changed from <none> to 715dc1bd8c2f4e85ad878fe61b6baf0a (127.26.250.131). New cstate: current_term: 1 leader_uuid: "715dc1bd8c2f4e85ad878fe61b6baf0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e7c358e873e4d91870d10a3afead870" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "715dc1bd8c2f4e85ad878fe61b6baf0a" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 44965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd434da39a154335ac952748eb1d1fbd" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 33883 } health_report { overall_health: UNKNOWN } } }
Aug 12 03:08:00 dist-test-slave-zpfg krb5kdc[28183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504078, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.129@KRBTEST.COM
I20260812 03:08:00.380967 28731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:00.364822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60912 (local address 127.26.250.129:33883)
0812 03:08:00.365046 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:00.365051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:00.365214 (+ 163us) server_negotiation.cc:407] Connection header received
0812 03:08:00.365421 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:00.365424 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:00.365481 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:00.365576 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:00.366502 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.367158 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.368199 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.368450 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.368578 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:00.376148 (+ 7570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:00.376167 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:00.376187 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:00.376225 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:00.378303 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:00.378917 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:00.378924 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:00.378926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:00.379020 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:00.379471 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:00.379478 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:00.379480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:00.379934 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:00.380387 (+ 453us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:00.380597 (+ 210us) server_negotiation.cc:299] Negotiation successful
0812 03:08:00.380735 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
Aug 12 03:08:00 dist-test-slave-zpfg krb5kdc[28183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504080, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:08:00 dist-test-slave-zpfg krb5kdc[28183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504080, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.250.129@KRBTEST.COM
I20260812 03:08:00.412428 28731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:00.403476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60926 (local address 127.26.250.129:33883)
0812 03:08:00.403612 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:00.403616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:00.403630 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:08:00.403676 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:00.403678 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:00.403720 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:00.403847 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:00.404551 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.405137 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.406014 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.406268 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.406432 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:00.408845 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:00.408848 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:00.408850 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:00.408875 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:00.410623 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:00.411055 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:00.411057 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:00.411059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:00.411137 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:00.411449 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:00.411451 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:00.411453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:00.411627 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:00.411764 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:00.412089 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 03:08:00.412291 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Aug 12 03:08:00 dist-test-slave-zpfg krb5kdc[28183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:08:00 dist-test-slave-zpfg krb5kdc[28183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504080, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.129@KRBTEST.COM
I20260812 03:08:00.443560 28731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:00.432922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60928 (local address 127.26.250.129:33883)
0812 03:08:00.433140 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:00.433144 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:00.433333 (+ 189us) server_negotiation.cc:407] Connection header received
0812 03:08:00.433464 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:00.433467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:00.433524 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:00.433633 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:00.434498 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.435085 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.436044 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:00.436318 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:00.436444 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:00.439310 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:00.439313 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:00.439315 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:00.439345 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:00.441323 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:00.441830 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:00.441834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:00.441835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:00.441931 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:00.442452 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:00.442456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:00.442458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:00.442675 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:00.442859 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:00.443175 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 03:08:00.443323 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
W20260812 03:08:00.445168 28343 tablet_service.cc:2908] Rejecting scan request for tablet 79f2720ea4764a6f94e6b494a16ef628: Uninitialized: clean time has not yet been initialized
I20260812 03:08:00.447741 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 28297
I20260812 03:08:00.456846 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 28440
I20260812 03:08:00.466028 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 28578
I20260812 03:08:00.473546 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 28196
2026-08-12T03:08:00Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2762 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:08:00 dist-test-slave-zpfg krb5kdc[28759](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:08:00 dist-test-slave-zpfg krb5kdc[28759](info): set up 2 sockets
Aug 12 03:08:00 dist-test-slave-zpfg krb5kdc[28759](info): commencing operation
krb5kdc: starting...
W20260812 03:08:01.667878 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:08:01 dist-test-slave-zpfg krb5kdc[28759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:08:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:08:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:08:01.810894 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:45553
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:42039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:45553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:01.935679 28774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:01.936034 28774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:01.936120 28774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:01.940369 28774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:08:01.940485 28774 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 03:08:01.940517 28774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:01.940562 28774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:01.940601 28774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:08:01.940637 28774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:08:01.946429 28774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:42039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:45553
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:45553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:01.947765 28774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:01.948915 28774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:01.958299 28780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:01.958299 28779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:01.958318 28774 server_base.cc:1034] running on GCE node
W20260812 03:08:01.958325 28782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:01.959138 28774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:01.960351 28774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:01.961560 28774 hybrid_clock.cc:648] HybridClock initialized: now 1786504081961538 us; error 40 us; skew 500 ppm
Aug 12 03:08:01 dist-test-slave-zpfg krb5kdc[28759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504081, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:01.964704 28774 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:08:01.965734 28774 webserver.cc:460] Webserver started at http://127.26.250.190:46725/ using document root <none> and password file <none>
I20260812 03:08:01.966408 28774 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:01.966507 28774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:01.966763 28774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:01.969017 28774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "8b7dde5739cb42b7bbf2c7184adc376e"
format_stamp: "Formatted at 2026-08-12 03:08:01 on dist-test-slave-zpfg"
server_key: "fb7ec814f9802388387e2bee12f45598"
server_key_iv: "c75b9c27ba9313bb396b41185e8bafa7"
server_key_version: "encryptionkey@0"
I20260812 03:08:01.969627 28774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "8b7dde5739cb42b7bbf2c7184adc376e"
format_stamp: "Formatted at 2026-08-12 03:08:01 on dist-test-slave-zpfg"
server_key: "fb7ec814f9802388387e2bee12f45598"
server_key_iv: "c75b9c27ba9313bb396b41185e8bafa7"
server_key_version: "encryptionkey@0"
I20260812 03:08:01.973768 28774 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:08:01.976527 28789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:01.977975 28774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:08:01.978132 28774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "8b7dde5739cb42b7bbf2c7184adc376e"
format_stamp: "Formatted at 2026-08-12 03:08:01 on dist-test-slave-zpfg"
server_key: "fb7ec814f9802388387e2bee12f45598"
server_key_iv: "c75b9c27ba9313bb396b41185e8bafa7"
server_key_version: "encryptionkey@0"
I20260812 03:08:01.978257 28774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:01.999951 28774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:02.003810 28774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:02.004055 28774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:02.014238 28774 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:45553
I20260812 03:08:02.014290 28855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:45553 every 8 connection(s)
I20260812 03:08:02.015390 28774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:08:02.018666 28856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:02.020171 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 28774
I20260812 03:08:02.020313 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:08:02.020579 27626 external_mini_cluster.cc:1428] Setting key d154e23ed3aa09a2125401c438de7fb2
I20260812 03:08:02.025177 28856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b7dde5739cb42b7bbf2c7184adc376e: Bootstrap starting.
I20260812 03:08:02.027633 28856 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b7dde5739cb42b7bbf2c7184adc376e: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:02.028441 28856 log.cc:824] T 00000000000000000000000000000000 P 8b7dde5739cb42b7bbf2c7184adc376e: Log is configured to *not* fsync() on all Append() calls
Aug 12 03:08:02 dist-test-slave-zpfg krb5kdc[28759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:02.031104 28856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b7dde5739cb42b7bbf2c7184adc376e: No bootstrap required, opened a new log
I20260812 03:08:02.034855 28856 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b7dde5739cb42b7bbf2c7184adc376e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7dde5739cb42b7bbf2c7184adc376e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45553 } }
I20260812 03:08:02.035177 28856 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b7dde5739cb42b7bbf2c7184adc376e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:02.035270 28856 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b7dde5739cb42b7bbf2c7184adc376e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b7dde5739cb42b7bbf2c7184adc376e, State: Initialized, Role: FOLLOWER
I20260812 03:08:02.035878 28856 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b7dde5739cb42b7bbf2c7184adc376e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7dde5739cb42b7bbf2c7184adc376e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45553 } }
I20260812 03:08:02.036069 28856 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b7dde5739cb42b7bbf2c7184adc376e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:08:02.036159 28856 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b7dde5739cb42b7bbf2c7184adc376e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:08:02.036247 28856 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b7dde5739cb42b7bbf2c7184adc376e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:02.037173 28859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:02.022211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56662 (local address 127.26.250.190:45553)
0812 03:08:02.022710 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:02.022726 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:02.022764 (+ 38us) server_negotiation.cc:407] Connection header received
0812 03:08:02.023450 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:02.023467 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:02.023828 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:02.024191 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:02.025046 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.026096 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.027065 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.027575 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.027762 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:02.030989 (+ 3227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:02.030994 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:02.031008 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:02.031036 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:02.033731 (+ 2695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:02.034410 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:02.034416 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:02.034421 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:02.034544 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:02.034866 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:02.034869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:02.034870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:02.035423 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:02.035594 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:02.035903 (+ 309us) server_negotiation.cc:299] Negotiation successful
0812 03:08:02.036259 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":151,"threads_started":1}
I20260812 03:08:02.037217 28856 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b7dde5739cb42b7bbf2c7184adc376e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7dde5739cb42b7bbf2c7184adc376e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45553 } }
I20260812 03:08:02.037636 28856 leader_election.cc:302] T 00000000000000000000000000000000 P 8b7dde5739cb42b7bbf2c7184adc376e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b7dde5739cb42b7bbf2c7184adc376e; no voters:
I20260812 03:08:02.037966 28856 leader_election.cc:288] T 00000000000000000000000000000000 P 8b7dde5739cb42b7bbf2c7184adc376e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:08:02.038089 28861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b7dde5739cb42b7bbf2c7184adc376e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:02.038389 28861 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b7dde5739cb42b7bbf2c7184adc376e [term 1 LEADER]: Becoming Leader. State: Replica: 8b7dde5739cb42b7bbf2c7184adc376e, State: Running, Role: LEADER
I20260812 03:08:02.038709 28861 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b7dde5739cb42b7bbf2c7184adc376e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7dde5739cb42b7bbf2c7184adc376e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45553 } }
I20260812 03:08:02.039300 28856 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b7dde5739cb42b7bbf2c7184adc376e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:08:02.039959 28863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b7dde5739cb42b7bbf2c7184adc376e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b7dde5739cb42b7bbf2c7184adc376e. Latest consensus state: current_term: 1 leader_uuid: "8b7dde5739cb42b7bbf2c7184adc376e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7dde5739cb42b7bbf2c7184adc376e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45553 } } }
I20260812 03:08:02.040107 28863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b7dde5739cb42b7bbf2c7184adc376e [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:02.040485 28862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b7dde5739cb42b7bbf2c7184adc376e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b7dde5739cb42b7bbf2c7184adc376e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7dde5739cb42b7bbf2c7184adc376e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45553 } } }
I20260812 03:08:02.040544 28869 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:08:02.040603 28862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b7dde5739cb42b7bbf2c7184adc376e [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:02.044203 28869 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:08:02.050529 28869 catalog_manager.cc:1347] Generated new cluster ID: abddadb28ccd4d5991f1da8ad7edc22c
I20260812 03:08:02.050631 28869 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:08:02.080755 28869 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:08:02.081923 28869 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:08:02.093523 28869 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b7dde5739cb42b7bbf2c7184adc376e: Generated new TSK 0
I20260812 03:08:02.094528 28869 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.250.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
I20260812 03:08:02.134792 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:0
--local_ip_for_outbound_sockets=127.26.250.129
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:45553
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:42039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:02.265584 28884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:02.265971 28884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:02.266036 28884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:02.270993 28884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:02.271085 28884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:02.271133 28884 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:08:02.271173 28884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:08:02.277822 28884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:42039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.250.190:45553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:02.279191 28884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:02.280411 28884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:02.292846 28890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:02.292897 28891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:02.292951 28893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:02.293226 28884 server_base.cc:1034] running on GCE node
I20260812 03:08:02.293710 28884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:02.294428 28884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:02.295679 28884 hybrid_clock.cc:648] HybridClock initialized: now 1786504082295657 us; error 36 us; skew 500 ppm
Aug 12 03:08:02 dist-test-slave-zpfg krb5kdc[28759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504082, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:02.299117 28884 init.cc:377] Logged in from keytab as kudu/127.26.250.129@KRBTEST.COM (short username kudu)
I20260812 03:08:02.300267 28884 webserver.cc:460] Webserver started at http://127.26.250.129:42243/ using document root <none> and password file <none>
I20260812 03:08:02.301092 28884 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:02.301213 28884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:02.301522 28884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:02.304244 28884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-0/data/instance:
uuid: "94ce0df0f78c4548a748fe7394171fa9"
format_stamp: "Formatted at 2026-08-12 03:08:02 on dist-test-slave-zpfg"
server_key: "1a525de972d8aac33f6f317bd7a23ecf"
server_key_iv: "6108d4923cd47b2d5e69d899b708e12f"
server_key_version: "encryptionkey@0"
I20260812 03:08:02.304977 28884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance:
uuid: "94ce0df0f78c4548a748fe7394171fa9"
format_stamp: "Formatted at 2026-08-12 03:08:02 on dist-test-slave-zpfg"
server_key: "1a525de972d8aac33f6f317bd7a23ecf"
server_key_iv: "6108d4923cd47b2d5e69d899b708e12f"
server_key_version: "encryptionkey@0"
I20260812 03:08:02.309792 28884 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:08:02.313040 28900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:02.314679 28884 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:08:02.314863 28884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "94ce0df0f78c4548a748fe7394171fa9"
format_stamp: "Formatted at 2026-08-12 03:08:02 on dist-test-slave-zpfg"
server_key: "1a525de972d8aac33f6f317bd7a23ecf"
server_key_iv: "6108d4923cd47b2d5e69d899b708e12f"
server_key_version: "encryptionkey@0"
I20260812 03:08:02.314997 28884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:02.332026 28884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:02.336504 28884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:02.336736 28884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:02.338430 28884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:02.338506 28884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:02.338580 28884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:02.338625 28884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:08:02 dist-test-slave-zpfg krb5kdc[28759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504082, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:02.350802 28884 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:34813
I20260812 03:08:02.351907 28884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:08:02.352671 29028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:34813 every 8 connection(s)
I20260812 03:08:02.354488 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 28884
I20260812 03:08:02.354616 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance
I20260812 03:08:02.355017 27626 external_mini_cluster.cc:1428] Setting key 307877c358f280e915451b51fd8814e5
I20260812 03:08:02.356204 28859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:02.339517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:56435 (local address 127.26.250.190:45553)
0812 03:08:02.339665 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:02.339669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:02.340923 (+ 1254us) server_negotiation.cc:407] Connection header received
0812 03:08:02.342398 (+ 1475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:02.342402 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:02.342469 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:02.342560 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:02.344014 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.345047 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.347188 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.347524 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.347630 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:02.351502 (+ 3872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:02.351510 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:02.351515 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:02.351552 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:02.353510 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:02.354106 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:02.354125 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:02.354127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:02.354209 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:02.354722 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:02.354727 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:02.354731 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:02.354990 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:02.355112 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:02.355890 (+ 778us) server_negotiation.cc:299] Negotiation successful
0812 03:08:02.356028 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:08:02.357296 28909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:02.340161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:45553 (local address 127.26.250.129:56435)
0812 03:08:02.340748 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:02.340799 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:02.342103 (+ 1304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:02.342759 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:02.342767 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:02.343140 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:02.343795 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:02.343814 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.345243 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.345249 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:02.347006 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:02.347023 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.347733 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.347750 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:02.348004 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:02.348974 (+ 970us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:02.349003 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:02.351260 (+ 2257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:02.353681 (+ 2421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:02.353687 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:02.353699 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:02.354001 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:02.354326 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:02.354330 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:02.354332 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:02.354598 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:02.355151 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:02.355159 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:02.355528 (+ 369us) client_negotiation.cc:769] Sending connection context
0812 03:08:02.355761 (+ 233us) client_negotiation.cc:240] Negotiation successful
0812 03:08:02.356100 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":151,"threads_started":1}
I20260812 03:08:02.358744 29029 heartbeater.cc:347] Connected to a master server at 127.26.250.190:45553
I20260812 03:08:02.358939 29029 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:02.359570 29029 heartbeater.cc:511] Master 127.26.250.190:45553 requested a full tablet report, sending...
I20260812 03:08:02.361456 28804 ts_manager.cc:194] Registered new tserver with Master: 94ce0df0f78c4548a748fe7394171fa9 (127.26.250.129:34813)
I20260812 03:08:02.363165 28804 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.129@KRBTEST.COM'} at 127.26.250.129:56435
WARNING: no policy specified for kudu/127.26.250.130@KRBTEST.COM; defaulting to no policy
I20260812 03:08:02.369894 28859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:02.361434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:45079 (local address 127.26.250.190:45553)
0812 03:08:02.361570 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:02.361575 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:02.361780 (+ 205us) server_negotiation.cc:407] Connection header received
0812 03:08:02.361939 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:02.361943 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:02.361998 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:02.362092 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:02.362714 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.363283 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.364431 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.364683 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.364800 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:02.366189 (+ 1389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:02.366194 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:02.366200 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:02.366390 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:02.368011 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:02.368508 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:02.368512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:02.368514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:02.368597 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:02.368982 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:02.368984 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:02.368986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:02.369177 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:02.369269 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:02.369658 (+ 389us) server_negotiation.cc:299] Negotiation successful
0812 03:08:02.369780 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:08:02.369907 29036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:02.361369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:45553 (local address 127.26.250.129:45079)
0812 03:08:02.361683 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:02.361697 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:02.361829 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:02.362120 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:02.362123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:02.362462 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:02.362583 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:02.362590 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.363424 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.363428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:02.364292 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:02.364302 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.364820 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.364824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:02.364963 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:02.365620 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:02.365634 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:02.366021 (+ 387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:02.368134 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:02.368139 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:02.368141 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:02.368389 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:02.368735 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:02.368739 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:02.368742 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:02.368868 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:02.369324 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:02.369326 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:02.369411 (+ 85us) client_negotiation.cc:769] Sending connection context
0812 03:08:02.369501 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 03:08:02.369615 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":3,"thread_start_us":123,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
I20260812 03:08:02.390568 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:0
--local_ip_for_outbound_sockets=127.26.250.130
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:45553
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:42039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:02.514072 29039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:02.514456 29039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:02.514565 29039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:02.519440 29039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:02.519555 29039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:02.519634 29039 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:08:02.519716 29039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:08:02.525545 29039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:42039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.250.190:45553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:02.526921 29039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:02.528090 29039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:02.539361 29044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:02.539526 29039 server_base.cc:1034] running on GCE node
W20260812 03:08:02.539395 29048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:02.539412 29045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:02.540138 29039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:02.541031 29039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:02.542248 29039 hybrid_clock.cc:648] HybridClock initialized: now 1786504082542216 us; error 50 us; skew 500 ppm
Aug 12 03:08:02 dist-test-slave-zpfg krb5kdc[28759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504082, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:02.545753 29039 init.cc:377] Logged in from keytab as kudu/127.26.250.130@KRBTEST.COM (short username kudu)
I20260812 03:08:02.546963 29039 webserver.cc:460] Webserver started at http://127.26.250.130:44777/ using document root <none> and password file <none>
I20260812 03:08:02.547766 29039 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:02.547883 29039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:02.548175 29039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:02.550848 29039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-1/data/instance:
uuid: "0294274610e94b7a84d2883851f97131"
format_stamp: "Formatted at 2026-08-12 03:08:02 on dist-test-slave-zpfg"
server_key: "d750c847d81660a1837a876215dbf369"
server_key_iv: "972dabf680151b58f3e7fe536f80b446"
server_key_version: "encryptionkey@0"
I20260812 03:08:02.551558 29039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance:
uuid: "0294274610e94b7a84d2883851f97131"
format_stamp: "Formatted at 2026-08-12 03:08:02 on dist-test-slave-zpfg"
server_key: "d750c847d81660a1837a876215dbf369"
server_key_iv: "972dabf680151b58f3e7fe536f80b446"
server_key_version: "encryptionkey@0"
I20260812 03:08:02.556222 29039 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:08:02.559466 29056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:02.561002 29039 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:08:02.561185 29039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "0294274610e94b7a84d2883851f97131"
format_stamp: "Formatted at 2026-08-12 03:08:02 on dist-test-slave-zpfg"
server_key: "d750c847d81660a1837a876215dbf369"
server_key_iv: "972dabf680151b58f3e7fe536f80b446"
server_key_version: "encryptionkey@0"
I20260812 03:08:02.561371 29039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:02.579743 29039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:02.583937 29039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:02.584187 29039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:02.585916 29039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:02.585996 29039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:02.586059 29039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:02.586110 29039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:08:02 dist-test-slave-zpfg krb5kdc[28759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504082, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:02.598278 29039 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:38727
I20260812 03:08:02.598431 29179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:38727 every 8 connection(s)
I20260812 03:08:02.599537 29039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:08:02.601142 28859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:02.586904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:42669 (local address 127.26.250.190:45553)
0812 03:08:02.587074 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:02.587079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:02.588033 (+ 954us) server_negotiation.cc:407] Connection header received
0812 03:08:02.589174 (+ 1141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:02.589178 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:02.589248 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:02.589334 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:02.590930 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.591618 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.593158 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.593398 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.593526 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:02.596866 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:02.596869 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:02.596872 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:02.596902 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:02.598721 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:02.599362 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:02.599366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:02.599368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:02.599502 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:02.599961 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:02.599970 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:02.599989 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:02.600245 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:02.600361 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:02.600903 (+ 542us) server_negotiation.cc:299] Negotiation successful
0812 03:08:02.601019 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 03:08:02.602026 29066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:02.587357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:45553 (local address 127.26.250.130:42669)
0812 03:08:02.587831 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:02.587897 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:02.588877 (+ 980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:02.589617 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:02.589628 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:02.590097 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:02.590759 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:02.590776 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.591759 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.591763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:02.593010 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:02.593022 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.593588 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.593593 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:02.593849 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:02.594618 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:02.594648 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:02.596652 (+ 2004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:02.598897 (+ 2245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:02.598902 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:02.598935 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:02.599241 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:02.599641 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:02.599645 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:02.599647 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:02.599809 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:02.600424 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:02.600429 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:02.600660 (+ 231us) client_negotiation.cc:769] Sending connection context
0812 03:08:02.600823 (+ 163us) client_negotiation.cc:240] Negotiation successful
0812 03:08:02.601085 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":137,"threads_started":1}
I20260812 03:08:02.603439 29180 heartbeater.cc:347] Connected to a master server at 127.26.250.190:45553
I20260812 03:08:02.603603 29180 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:02.604332 29180 heartbeater.cc:511] Master 127.26.250.190:45553 requested a full tablet report, sending...
I20260812 03:08:02.605602 28803 ts_manager.cc:194] Registered new tserver with Master: 0294274610e94b7a84d2883851f97131 (127.26.250.130:38727)
I20260812 03:08:02.606477 28803 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.130@KRBTEST.COM'} at 127.26.250.130:42669
I20260812 03:08:02.609440 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 29039
I20260812 03:08:02.609572 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance
I20260812 03:08:02.609980 27626 external_mini_cluster.cc:1428] Setting key fd7ae26df23c4a8ba950ad483ff1d943
I20260812 03:08:02.614830 29186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:02.605982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:45553 (local address 127.26.250.130:53795)
0812 03:08:02.606320 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:02.606335 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:02.606480 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:02.606786 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:02.606789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:02.607066 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:02.607208 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:02.607214 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.608315 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.608317 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:02.608891 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:02.608899 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.609450 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.609457 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:02.609591 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:02.610193 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:02.610210 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:02.610745 (+ 535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:02.612791 (+ 2046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:02.612798 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:02.612804 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:02.613122 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:02.613409 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:02.613415 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:02.613420 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:02.613539 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:02.614169 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:02.614176 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:02.614383 (+ 207us) client_negotiation.cc:769] Sending connection context
0812 03:08:02.614520 (+ 137us) client_negotiation.cc:240] Negotiation successful
0812 03:08:02.614664 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":115,"threads_started":1}
I20260812 03:08:02.614957 28859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:02.606105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:53795 (local address 127.26.250.190:45553)
0812 03:08:02.606237 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:02.606242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:02.606441 (+ 199us) server_negotiation.cc:407] Connection header received
0812 03:08:02.606600 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:02.606604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:02.606662 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:02.606730 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:02.607330 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.608094 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.609004 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.609235 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.609316 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:02.610903 (+ 1587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:02.610907 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:02.610910 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:02.610938 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:02.612662 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:02.613228 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:02.613231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:02.613232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:02.613309 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:02.613685 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:02.613688 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:02.613689 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:02.613934 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:02.614179 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:02.614610 (+ 431us) server_negotiation.cc:299] Negotiation successful
0812 03:08:02.614756 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
WARNING: no policy specified for kudu/127.26.250.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
I20260812 03:08:02.642077 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:0
--local_ip_for_outbound_sockets=127.26.250.131
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:45553
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:42039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:02.763185 29189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:02.763485 29189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:02.763533 29189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:02.768049 29189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:02.768139 29189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:02.768190 29189 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:08:02.768239 29189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:08:02.774245 29189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:42039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.250.190:45553
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:02.775540 29189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:02.776695 29189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:02.788959 29195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:02.789247 29189 server_base.cc:1034] running on GCE node
W20260812 03:08:02.788959 29197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:02.789052 29194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:02.789875 29189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:02.790688 29189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:02.791941 29189 hybrid_clock.cc:648] HybridClock initialized: now 1786504082791922 us; error 32 us; skew 500 ppm
Aug 12 03:08:02 dist-test-slave-zpfg krb5kdc[28759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504082, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:02.795104 29189 init.cc:377] Logged in from keytab as kudu/127.26.250.131@KRBTEST.COM (short username kudu)
I20260812 03:08:02.796181 29189 webserver.cc:460] Webserver started at http://127.26.250.131:32783/ using document root <none> and password file <none>
I20260812 03:08:02.796847 29189 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:02.796973 29189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:02.797231 29189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:02.799620 29189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-2/data/instance:
uuid: "ecbf547cfa88443ba4f9ac8325dffa5b"
format_stamp: "Formatted at 2026-08-12 03:08:02 on dist-test-slave-zpfg"
server_key: "750baf0010fdca696b3cd86d3af228a1"
server_key_iv: "320898467f4ebc19b7a337dad744493a"
server_key_version: "encryptionkey@0"
I20260812 03:08:02.800295 29189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance:
uuid: "ecbf547cfa88443ba4f9ac8325dffa5b"
format_stamp: "Formatted at 2026-08-12 03:08:02 on dist-test-slave-zpfg"
server_key: "750baf0010fdca696b3cd86d3af228a1"
server_key_iv: "320898467f4ebc19b7a337dad744493a"
server_key_version: "encryptionkey@0"
I20260812 03:08:02.804714 29189 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:08:02.807641 29204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:02.809119 29189 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:08:02.809382 29189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "ecbf547cfa88443ba4f9ac8325dffa5b"
format_stamp: "Formatted at 2026-08-12 03:08:02 on dist-test-slave-zpfg"
server_key: "750baf0010fdca696b3cd86d3af228a1"
server_key_iv: "320898467f4ebc19b7a337dad744493a"
server_key_version: "encryptionkey@0"
I20260812 03:08:02.809522 29189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:02.821893 29189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:02.825619 29189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:02.825888 29189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:02.827572 29189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:02.827651 29189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:02.827732 29189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:02.827778 29189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:08:02 dist-test-slave-zpfg krb5kdc[28759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504082, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:02.839665 29189 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:35227
I20260812 03:08:02.840308 29328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:35227 every 8 connection(s)
I20260812 03:08:02.841280 29189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:08:02.844164 28859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:02.828415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:33259 (local address 127.26.250.190:45553)
0812 03:08:02.828609 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:02.828616 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:02.829594 (+ 978us) server_negotiation.cc:407] Connection header received
0812 03:08:02.830714 (+ 1120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:02.830718 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:02.830787 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:02.830853 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:02.832396 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.833331 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.834741 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.835036 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.835208 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:02.838996 (+ 3788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:02.839001 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:02.839003 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:02.839035 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:02.841691 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:02.842555 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:02.842559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:02.842560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:02.842652 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:02.843041 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:02.843044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:02.843045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:02.843244 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:02.843364 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:02.843920 (+ 556us) server_negotiation.cc:299] Negotiation successful
0812 03:08:02.844038 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 03:08:02.845247 29213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:02.828826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:45553 (local address 127.26.250.131:33259)
0812 03:08:02.829401 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:02.829457 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:02.830450 (+ 993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:02.831228 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:02.831244 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:02.831587 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:02.832091 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:02.832133 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.833486 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.833490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:02.834592 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:02.834603 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.835260 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.835268 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:02.835539 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:02.836327 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:02.836375 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:02.838742 (+ 2367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:02.841855 (+ 3113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:02.841863 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:02.841880 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:02.842438 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:02.842776 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:02.842779 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:02.842780 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:02.842930 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:02.843373 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:02.843379 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:02.843637 (+ 258us) client_negotiation.cc:769] Sending connection context
0812 03:08:02.843839 (+ 202us) client_negotiation.cc:240] Negotiation successful
0812 03:08:02.844132 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":129,"threads_started":1}
I20260812 03:08:02.846593 29329 heartbeater.cc:347] Connected to a master server at 127.26.250.190:45553
I20260812 03:08:02.846809 29329 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:02.847450 29329 heartbeater.cc:511] Master 127.26.250.190:45553 requested a full tablet report, sending...
I20260812 03:08:02.848738 28804 ts_manager.cc:194] Registered new tserver with Master: ecbf547cfa88443ba4f9ac8325dffa5b (127.26.250.131:35227)
I20260812 03:08:02.849694 28804 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.131@KRBTEST.COM'} at 127.26.250.131:33259
I20260812 03:08:02.850368 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 29189
I20260812 03:08:02.850514 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance
I20260812 03:08:02.850812 27626 external_mini_cluster.cc:1428] Setting key 5f21852a3ad7e0434116f24710d8028b
I20260812 03:08:02.853140 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:08:02.859230 28859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:02.849375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:48643 (local address 127.26.250.190:45553)
0812 03:08:02.849520 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:02.849525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:02.849649 (+ 124us) server_negotiation.cc:407] Connection header received
0812 03:08:02.849861 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:02.849864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:02.849917 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:02.850002 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:02.852246 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.852769 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.853813 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.854043 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.854160 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:02.855470 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:02.855477 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:02.855482 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:02.855519 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:02.857317 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:02.857802 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:02.857805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:02.857807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:02.857902 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:02.858228 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:02.858231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:02.858233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:02.858498 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:02.858613 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:02.858903 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 03:08:02.859034 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 03:08:02.859267 29336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:02.849219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:45553 (local address 127.26.250.131:48643)
0812 03:08:02.849563 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:02.849576 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:02.849699 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:02.850020 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:02.850024 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:02.850237 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:02.850393 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:02.852135 (+ 1742us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.852905 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.852908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:02.853665 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:02.853676 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.854167 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.854170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:02.854269 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:02.854928 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:02.854940 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:02.855316 (+ 376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:02.857457 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:02.857460 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:02.857463 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:02.857685 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:02.858004 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:02.858007 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:02.858009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:02.858121 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:02.858602 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:02.858605 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:02.858724 (+ 119us) client_negotiation.cc:769] Sending connection context
0812 03:08:02.858842 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 03:08:02.859038 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":143,"threads_started":1}
Aug 12 03:08:02 dist-test-slave-zpfg krb5kdc[28759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504082, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:08:02 dist-test-slave-zpfg krb5kdc[28759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504082, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:02.885370 28859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:02.875372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56670 (local address 127.26.250.190:45553)
0812 03:08:02.875542 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:02.875546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:02.875558 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:08:02.875600 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:02.875602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:02.875647 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:02.875739 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:02.876524 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.877069 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.877976 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.878230 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.878345 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:02.881392 (+ 3047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:02.881395 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:02.881397 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:02.881428 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:02.883395 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:02.883926 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:02.883936 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:02.883937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:02.884027 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:02.884349 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:02.884354 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:02.884355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:02.884572 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:02.884728 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:02.885024 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 03:08:02.885159 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 03:08:02.889247 28804 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56670:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:08:02.892670 28804 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:08:02.909929 29348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:02.903498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:38727 (local address 127.0.0.1:50036)
0812 03:08:02.904778 (+ 1280us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:02.904794 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:02.904904 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:02.905644 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:02.905649 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:02.905673 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:02.905785 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:02.905792 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.907015 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.907018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:02.908211 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:02.908221 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.909249 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.909253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:02.909399 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:02.909446 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 03:08:02.909582 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 03:08:02.909732 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1197,"mutex_wait_us":11,"spinlock_wait_cycles":3328,"thread_start_us":151,"threads_started":1}
I20260812 03:08:02.909945 29351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:02.904316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50036 (local address 127.26.250.130:38727)
0812 03:08:02.905171 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:02.905184 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:02.905212 (+ 28us) server_negotiation.cc:407] Connection header received
0812 03:08:02.905293 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:02.905298 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:02.905482 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:02.905642 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:02.905915 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.906886 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.908343 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.909106 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.909264 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:02.909411 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:02.909636 (+ 225us) server_negotiation.cc:299] Negotiation successful
0812 03:08:02.909762 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":158,"threads_started":1}
I20260812 03:08:02.910158 29347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:02.903408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:34813 (local address 127.0.0.1:37958)
0812 03:08:02.904061 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:02.904127 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:02.904268 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:02.905316 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:02.905323 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:02.905350 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:02.905527 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:02.905535 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.907284 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.907288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:02.908654 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:02.908666 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.909779 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.909782 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:02.909882 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:02.909907 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 03:08:02.909960 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 03:08:02.910040 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":100,"threads_started":1}
I20260812 03:08:02.911069 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:02.904247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:35227 (local address 127.0.0.1:49644)
0812 03:08:02.904778 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:02.904794 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:02.904904 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:02.905820 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:02.905822 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:02.905838 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:02.905971 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:02.905981 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.907596 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.907599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:02.908878 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:02.908886 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.910315 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.910321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:02.910690 (+ 369us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:02.910710 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 03:08:02.910823 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 03:08:02.910902 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":52,"spinlock_wait_cycles":6656,"thread_start_us":112,"threads_started":1}
I20260812 03:08:02.911298 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:02.903744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37958 (local address 127.26.250.129:34813)
0812 03:08:02.904721 (+ 977us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:02.904729 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:02.904758 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:08:02.904855 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:02.904866 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:02.905136 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:02.905330 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:02.905711 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.907116 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.908806 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.909622 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.910863 (+ 1241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:02.910995 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:02.911050 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 03:08:02.911149 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":832,"thread_start_us":92,"threads_started":1}
I20260812 03:08:02.911650 29352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:02.904732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49644 (local address 127.26.250.131:35227)
0812 03:08:02.905198 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:02.905215 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:02.905236 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:08:02.905303 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:02.905307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:02.905482 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:02.905642 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:02.906136 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.907456 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.909018 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:02.909929 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:02.910461 (+ 532us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:02.910590 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:02.911366 (+ 776us) server_negotiation.cc:299] Negotiation successful
0812 03:08:02.911481 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":129,"threads_started":1}
I20260812 03:08:02.913973 28958 tablet_service.cc:1467] Processing CreateTablet for tablet 3e3c3e7ec7954b8aad5bbc9c8905a118 (DEFAULT_TABLE table=test-table [id=9567799769db42eea7945e85c22a09f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:02.913961 29109 tablet_service.cc:1467] Processing CreateTablet for tablet 3e3c3e7ec7954b8aad5bbc9c8905a118 (DEFAULT_TABLE table=test-table [id=9567799769db42eea7945e85c22a09f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:02.915172 28958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e3c3e7ec7954b8aad5bbc9c8905a118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:02.915221 29109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e3c3e7ec7954b8aad5bbc9c8905a118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:02.914831 29256 tablet_service.cc:1467] Processing CreateTablet for tablet 3e3c3e7ec7954b8aad5bbc9c8905a118 (DEFAULT_TABLE table=test-table [id=9567799769db42eea7945e85c22a09f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:02.915974 29256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e3c3e7ec7954b8aad5bbc9c8905a118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:02.922106 29353 tablet_bootstrap.cc:492] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 94ce0df0f78c4548a748fe7394171fa9: Bootstrap starting.
I20260812 03:08:02.924588 29353 tablet_bootstrap.cc:654] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 94ce0df0f78c4548a748fe7394171fa9: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:02.924873 29355 tablet_bootstrap.cc:492] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P ecbf547cfa88443ba4f9ac8325dffa5b: Bootstrap starting.
I20260812 03:08:02.926072 29353 log.cc:824] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 94ce0df0f78c4548a748fe7394171fa9: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:02.926719 29354 tablet_bootstrap.cc:492] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131: Bootstrap starting.
I20260812 03:08:02.927309 29355 tablet_bootstrap.cc:654] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P ecbf547cfa88443ba4f9ac8325dffa5b: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:02.928081 29355 log.cc:824] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P ecbf547cfa88443ba4f9ac8325dffa5b: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:02.928508 29353 tablet_bootstrap.cc:492] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 94ce0df0f78c4548a748fe7394171fa9: No bootstrap required, opened a new log
I20260812 03:08:02.928736 29353 ts_tablet_manager.cc:1397] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 94ce0df0f78c4548a748fe7394171fa9: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 03:08:02.929030 29354 tablet_bootstrap.cc:654] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:02.929899 29354 log.cc:824] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:02.930310 29355 tablet_bootstrap.cc:492] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P ecbf547cfa88443ba4f9ac8325dffa5b: No bootstrap required, opened a new log
I20260812 03:08:02.930584 29355 ts_tablet_manager.cc:1397] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P ecbf547cfa88443ba4f9ac8325dffa5b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.002s
I20260812 03:08:02.932333 29354 tablet_bootstrap.cc:492] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131: No bootstrap required, opened a new log
I20260812 03:08:02.932559 29354 ts_tablet_manager.cc:1397] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 03:08:02.933866 29353 raft_consensus.cc:348] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 94ce0df0f78c4548a748fe7394171fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:02.934185 29353 raft_consensus.cc:374] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 94ce0df0f78c4548a748fe7394171fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:02.934269 29353 raft_consensus.cc:729] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 94ce0df0f78c4548a748fe7394171fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94ce0df0f78c4548a748fe7394171fa9, State: Initialized, Role: FOLLOWER
I20260812 03:08:02.934131 29355 raft_consensus.cc:348] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P ecbf547cfa88443ba4f9ac8325dffa5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:02.934484 29355 raft_consensus.cc:374] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P ecbf547cfa88443ba4f9ac8325dffa5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:02.934576 29355 raft_consensus.cc:729] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P ecbf547cfa88443ba4f9ac8325dffa5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecbf547cfa88443ba4f9ac8325dffa5b, State: Initialized, Role: FOLLOWER
I20260812 03:08:02.934844 29353 consensus_queue.cc:260] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 94ce0df0f78c4548a748fe7394171fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:02.935065 29355 consensus_queue.cc:260] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P ecbf547cfa88443ba4f9ac8325dffa5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:02.935985 29029 heartbeater.cc:503] Master 127.26.250.190:45553 was elected leader, sending a full tablet report...
I20260812 03:08:02.936226 29355 ts_tablet_manager.cc:1428] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P ecbf547cfa88443ba4f9ac8325dffa5b: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260812 03:08:02.936596 29329 heartbeater.cc:503] Master 127.26.250.190:45553 was elected leader, sending a full tablet report...
I20260812 03:08:02.936695 29354 raft_consensus.cc:348] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:02.936970 29354 raft_consensus.cc:374] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:02.937031 29354 raft_consensus.cc:729] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0294274610e94b7a84d2883851f97131, State: Initialized, Role: FOLLOWER
I20260812 03:08:02.937531 29353 ts_tablet_manager.cc:1428] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 94ce0df0f78c4548a748fe7394171fa9: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260812 03:08:02.937520 29354 consensus_queue.cc:260] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:02.939112 29180 heartbeater.cc:503] Master 127.26.250.190:45553 was elected leader, sending a full tablet report...
I20260812 03:08:02.939458 29354 ts_tablet_manager.cc:1428] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131: Time spent starting tablet: real 0.007s user 0.005s sys 0.002s
I20260812 03:08:03.007063 29362 raft_consensus.cc:482] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:08:03.007310 29362 raft_consensus.cc:504] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:03.008783 29362 leader_election.cc:288] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94ce0df0f78c4548a748fe7394171fa9 (127.26.250.129:34813), ecbf547cfa88443ba4f9ac8325dffa5b (127.26.250.131:35227)
I20260812 03:08:03.013144 29359 raft_consensus.cc:482] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 94ce0df0f78c4548a748fe7394171fa9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:08:03.013693 29359 raft_consensus.cc:504] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 94ce0df0f78c4548a748fe7394171fa9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:03.014737 29066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:03.008991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:34813 (local address 127.26.250.130:60479)
0812 03:08:03.009154 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:03.009171 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:03.009284 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:03.009672 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:03.009677 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:03.009729 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:03.009905 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:03.009914 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.011283 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.011287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:03.012768 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:03.012779 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.014191 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.014195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:03.014365 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:03.014393 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 03:08:03.014475 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 03:08:03.014571 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"spinlock_wait_cycles":23040}
I20260812 03:08:03.015278 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:03.009160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:60479 (local address 127.26.250.129:34813)
0812 03:08:03.009386 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:03.009391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:03.009409 (+ 18us) server_negotiation.cc:407] Connection header received
0812 03:08:03.009474 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:03.009479 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:03.009533 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:03.009624 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:03.010105 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.011123 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.012951 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.013928 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.014062 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:03.014122 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:03.015057 (+ 935us) server_negotiation.cc:299] Negotiation successful
0812 03:08:03.015160 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260812 03:08:03.015714 29359 leader_election.cc:288] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 94ce0df0f78c4548a748fe7394171fa9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0294274610e94b7a84d2883851f97131 (127.26.250.130:38727), ecbf547cfa88443ba4f9ac8325dffa5b (127.26.250.131:35227)
I20260812 03:08:03.015893 28980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e3c3e7ec7954b8aad5bbc9c8905a118" candidate_uuid: "0294274610e94b7a84d2883851f97131" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94ce0df0f78c4548a748fe7394171fa9" is_pre_election: true
I20260812 03:08:03.016016 29352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:03.009473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:44713 (local address 127.26.250.131:35227)
0812 03:08:03.009665 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:03.009670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:03.009686 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:08:03.009879 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:03.009883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:03.009944 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:03.010037 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:03.011121 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.012042 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.013765 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.014521 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.014616 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:03.014666 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:03.015817 (+ 1151us) server_negotiation.cc:299] Negotiation successful
0812 03:08:03.015911 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 03:08:03.016145 28980 raft_consensus.cc:2454] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 94ce0df0f78c4548a748fe7394171fa9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0294274610e94b7a84d2883851f97131 in term 0.
I20260812 03:08:03.016702 29363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:03.009170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:35227 (local address 127.26.250.130:44713)
0812 03:08:03.009591 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:03.009607 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:03.009731 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:03.010096 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:03.010099 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:03.010120 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:03.010953 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:03.010964 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.012199 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.012202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:03.013602 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:03.013615 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.015529 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.015532 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:03.015641 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:03.015679 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 03:08:03.016413 (+ 734us) client_negotiation.cc:240] Negotiation successful
0812 03:08:03.016530 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":12,"spinlock_wait_cycles":12544,"thread_start_us":199,"threads_started":1}
I20260812 03:08:03.016672 29058 leader_election.cc:302] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0294274610e94b7a84d2883851f97131, 94ce0df0f78c4548a748fe7394171fa9; no voters:
I20260812 03:08:03.017230 29362 raft_consensus.cc:2781] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:08:03.017328 29362 raft_consensus.cc:482] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:08:03.017385 29362 raft_consensus.cc:3037] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:03.017985 29279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e3c3e7ec7954b8aad5bbc9c8905a118" candidate_uuid: "0294274610e94b7a84d2883851f97131" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" is_pre_election: true
I20260812 03:08:03.018397 29279 raft_consensus.cc:2454] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P ecbf547cfa88443ba4f9ac8325dffa5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0294274610e94b7a84d2883851f97131 in term 0.
I20260812 03:08:03.018965 29362 raft_consensus.cc:504] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:03.019362 29362 leader_election.cc:288] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131 [CANDIDATE]: Term 1 election: Requested vote from peers 94ce0df0f78c4548a748fe7394171fa9 (127.26.250.129:34813), ecbf547cfa88443ba4f9ac8325dffa5b (127.26.250.131:35227)
I20260812 03:08:03.019827 29279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e3c3e7ec7954b8aad5bbc9c8905a118" candidate_uuid: "0294274610e94b7a84d2883851f97131" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b"
I20260812 03:08:03.019964 29279 raft_consensus.cc:3037] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P ecbf547cfa88443ba4f9ac8325dffa5b [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:03.021694 29279 raft_consensus.cc:2454] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P ecbf547cfa88443ba4f9ac8325dffa5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0294274610e94b7a84d2883851f97131 in term 1.
I20260812 03:08:03.022086 29057 leader_election.cc:302] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0294274610e94b7a84d2883851f97131, ecbf547cfa88443ba4f9ac8325dffa5b; no voters:
I20260812 03:08:03.022324 29362 raft_consensus.cc:2781] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:03.022583 29362 raft_consensus.cc:686] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131 [term 1 LEADER]: Becoming Leader. State: Replica: 0294274610e94b7a84d2883851f97131, State: Running, Role: LEADER
I20260812 03:08:03.022678 28980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e3c3e7ec7954b8aad5bbc9c8905a118" candidate_uuid: "0294274610e94b7a84d2883851f97131" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94ce0df0f78c4548a748fe7394171fa9"
I20260812 03:08:03.022830 28980 raft_consensus.cc:3037] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 94ce0df0f78c4548a748fe7394171fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:03.023000 29362 consensus_queue.cc:237] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:03.023249 29351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:03.017792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:48311 (local address 127.26.250.130:38727)
0812 03:08:03.017927 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:03.017932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:03.017947 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:08:03.017993 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:03.017998 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:03.018052 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:03.018164 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:03.018615 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.019318 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.021096 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.021747 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.021848 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:03.021909 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:03.023032 (+ 1123us) server_negotiation.cc:299] Negotiation successful
0812 03:08:03.023141 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:08:03.023887 29364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:03.016755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:38727 (local address 127.26.250.129:48311)
0812 03:08:03.017481 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:03.017500 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:03.017611 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:03.018241 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:03.018244 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:03.018265 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:03.018424 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:03.018432 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.019537 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.019540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:03.020915 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:03.020931 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.022060 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.022063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:03.022773 (+ 710us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:03.022863 (+ 90us) client_negotiation.cc:769] Sending connection context
0812 03:08:03.023585 (+ 722us) client_negotiation.cc:240] Negotiation successful
0812 03:08:03.023695 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"mutex_wait_us":509,"thread_start_us":146,"threads_started":1}
I20260812 03:08:03.024289 28980 raft_consensus.cc:2454] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 94ce0df0f78c4548a748fe7394171fa9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0294274610e94b7a84d2883851f97131 in term 1.
I20260812 03:08:03.024680 29131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e3c3e7ec7954b8aad5bbc9c8905a118" candidate_uuid: "94ce0df0f78c4548a748fe7394171fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0294274610e94b7a84d2883851f97131" is_pre_election: true
I20260812 03:08:03.026160 29352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:03.017972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:45841 (local address 127.26.250.131:35227)
0812 03:08:03.018079 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:03.018083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:03.019440 (+ 1357us) server_negotiation.cc:407] Connection header received
0812 03:08:03.019617 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:03.019620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:03.019667 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:03.019714 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:03.020181 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.021004 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.024746 (+ 3742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.025373 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.025475 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:03.025527 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:03.025941 (+ 414us) server_negotiation.cc:299] Negotiation successful
0812 03:08:03.026039 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 03:08:03.026784 29365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:03.017154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:35227 (local address 127.26.250.129:45841)
0812 03:08:03.019333 (+ 2179us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:03.019362 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:03.019504 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:03.019916 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:03.019919 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:03.019937 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:03.020058 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:03.020064 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.021166 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.021169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:03.024589 (+ 3420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:03.024602 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.025639 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.025643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:03.025760 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:03.025786 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 03:08:03.025885 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 03:08:03.025966 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2103,"mutex_wait_us":4,"thread_start_us":94,"threads_started":1}
I20260812 03:08:03.027235 29279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e3c3e7ec7954b8aad5bbc9c8905a118" candidate_uuid: "94ce0df0f78c4548a748fe7394171fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" is_pre_election: true
I20260812 03:08:03.027431 29279 raft_consensus.cc:2379] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P ecbf547cfa88443ba4f9ac8325dffa5b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 94ce0df0f78c4548a748fe7394171fa9 in current term 1: Already voted for candidate 0294274610e94b7a84d2883851f97131 in this term.
I20260812 03:08:03.027634 28802 catalog_manager.cc:5495] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0294274610e94b7a84d2883851f97131 (127.26.250.130). New cstate: current_term: 1 leader_uuid: "0294274610e94b7a84d2883851f97131" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } health_report { overall_health: UNKNOWN } } }
I20260812 03:08:03.028020 28901 leader_election.cc:302] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 94ce0df0f78c4548a748fe7394171fa9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94ce0df0f78c4548a748fe7394171fa9; no voters: 0294274610e94b7a84d2883851f97131, ecbf547cfa88443ba4f9ac8325dffa5b
I20260812 03:08:03.028352 29359 raft_consensus.cc:2726] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 94ce0df0f78c4548a748fe7394171fa9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 03:08:03.073491 29351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:03.068798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50044 (local address 127.26.250.130:38727)
0812 03:08:03.068977 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:03.068982 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:03.068999 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:08:03.069370 (+ 371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:03.069373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:03.069445 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:03.069587 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:08:03.069917 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.070554 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.071464 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.071709 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.071820 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:03.072203 (+ 383us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:08:03.072774 (+ 571us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:08:03.072908 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:03.073086 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 03:08:03.073184 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 03:08:03.080053 29279 raft_consensus.cc:1261] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P ecbf547cfa88443ba4f9ac8325dffa5b [term 1 FOLLOWER]: Refusing update from remote peer 0294274610e94b7a84d2883851f97131: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:03.080101 28980 raft_consensus.cc:1261] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 94ce0df0f78c4548a748fe7394171fa9 [term 1 FOLLOWER]: Refusing update from remote peer 0294274610e94b7a84d2883851f97131: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:03.080837 29366 consensus_queue.cc:1035] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:03.081067 29362 consensus_queue.cc:1035] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:03.090899 29371 mvcc.cc:205] Tried to move back new op lower bound from 7317520724287676416 to 7317520724065890304. Current Snapshot: MvccSnapshot[applied={T|T < 7317520724287676416}]
I20260812 03:08:03.102849 28804 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:03.103157 28804 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:03.106535 28804 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b7dde5739cb42b7bbf2c7184adc376e: Sending DeleteTablet for 3 replicas of tablet 3e3c3e7ec7954b8aad5bbc9c8905a118
I20260812 03:08:03.107512 29109 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e3c3e7ec7954b8aad5bbc9c8905a118 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:03 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:50036
I20260812 03:08:03.107558 29256 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e3c3e7ec7954b8aad5bbc9c8905a118 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:03 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:49644
I20260812 03:08:03.107632 28958 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e3c3e7ec7954b8aad5bbc9c8905a118 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:03 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:37958
I20260812 03:08:03.108083 29380 tablet_replica.cc:330] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P ecbf547cfa88443ba4f9ac8325dffa5b: stopping tablet replica
I20260812 03:08:03.108083 29379 tablet_replica.cc:330] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131: stopping tablet replica
I20260812 03:08:03.108454 29379 raft_consensus.cc:2229] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:08:03.108453 29380 raft_consensus.cc:2229] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P ecbf547cfa88443ba4f9ac8325dffa5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:03.108693 28804 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56670:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 03:08:03.108807 29380 raft_consensus.cc:2258] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P ecbf547cfa88443ba4f9ac8325dffa5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:03.108846 29379 raft_consensus.cc:2258] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 03:08:03.109134 28804 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:08:03.109606 29381 tablet_replica.cc:330] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 94ce0df0f78c4548a748fe7394171fa9: stopping tablet replica
I20260812 03:08:03.109946 29381 raft_consensus.cc:2229] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 94ce0df0f78c4548a748fe7394171fa9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:03.110263 29381 raft_consensus.cc:2258] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 94ce0df0f78c4548a748fe7394171fa9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:03.110656 29379 ts_tablet_manager.cc:1905] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:03.111759 29381 ts_tablet_manager.cc:1905] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 94ce0df0f78c4548a748fe7394171fa9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:03.113966 29379 ts_tablet_manager.cc:1918] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:08:03.114079 29379 log.cc:1196] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-1/wal/wals/3e3c3e7ec7954b8aad5bbc9c8905a118
I20260812 03:08:03.114480 29379 ts_tablet_manager.cc:1939] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 0294274610e94b7a84d2883851f97131: Deleting consensus metadata
I20260812 03:08:03.115094 29381 ts_tablet_manager.cc:1918] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 94ce0df0f78c4548a748fe7394171fa9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:08:03.115216 29381 log.cc:1196] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 94ce0df0f78c4548a748fe7394171fa9: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-0/wal/wals/3e3c3e7ec7954b8aad5bbc9c8905a118
I20260812 03:08:03.115554 29381 ts_tablet_manager.cc:1939] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P 94ce0df0f78c4548a748fe7394171fa9: Deleting consensus metadata
I20260812 03:08:03.115939 29109 tablet_service.cc:1467] Processing CreateTablet for tablet 2ce1b4634fdf4d4a8ea59c2c4ce98d51 (DEFAULT_TABLE table=test-table [id=3c6950ef88c946f393eab6294e19c25a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:03.116178 29256 tablet_service.cc:1467] Processing CreateTablet for tablet 2ce1b4634fdf4d4a8ea59c2c4ce98d51 (DEFAULT_TABLE table=test-table [id=3c6950ef88c946f393eab6294e19c25a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:03.116250 29109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ce1b4634fdf4d4a8ea59c2c4ce98d51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:03.116582 28958 tablet_service.cc:1467] Processing CreateTablet for tablet 2ce1b4634fdf4d4a8ea59c2c4ce98d51 (DEFAULT_TABLE table=test-table [id=3c6950ef88c946f393eab6294e19c25a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:03.116647 29256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ce1b4634fdf4d4a8ea59c2c4ce98d51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:03.116866 28958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ce1b4634fdf4d4a8ea59c2c4ce98d51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:03.118902 29354 tablet_bootstrap.cc:492] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131: Bootstrap starting.
I20260812 03:08:03.119467 29355 tablet_bootstrap.cc:492] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P ecbf547cfa88443ba4f9ac8325dffa5b: Bootstrap starting.
I20260812 03:08:03.119977 29354 tablet_bootstrap.cc:654] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:03.120301 29380 ts_tablet_manager.cc:1905] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P ecbf547cfa88443ba4f9ac8325dffa5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:03.120452 29355 tablet_bootstrap.cc:654] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P ecbf547cfa88443ba4f9ac8325dffa5b: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:03.120868 28790 catalog_manager.cc:4841] TS 0294274610e94b7a84d2883851f97131 (127.26.250.130:38727): tablet 3e3c3e7ec7954b8aad5bbc9c8905a118 (table test-table [id=9567799769db42eea7945e85c22a09f0]) successfully deleted
I20260812 03:08:03.121127 29353 tablet_bootstrap.cc:492] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 94ce0df0f78c4548a748fe7394171fa9: Bootstrap starting.
I20260812 03:08:03.121308 28791 catalog_manager.cc:4841] TS 94ce0df0f78c4548a748fe7394171fa9 (127.26.250.129:34813): tablet 3e3c3e7ec7954b8aad5bbc9c8905a118 (table test-table [id=9567799769db42eea7945e85c22a09f0]) successfully deleted
I20260812 03:08:03.122277 29353 tablet_bootstrap.cc:654] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 94ce0df0f78c4548a748fe7394171fa9: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:03.122828 29354 tablet_bootstrap.cc:492] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131: No bootstrap required, opened a new log
I20260812 03:08:03.122927 29354 ts_tablet_manager.cc:1397] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260812 03:08:03.122982 29380 ts_tablet_manager.cc:1918] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P ecbf547cfa88443ba4f9ac8325dffa5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:08:03.123100 29380 log.cc:1196] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P ecbf547cfa88443ba4f9ac8325dffa5b: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/3e3c3e7ec7954b8aad5bbc9c8905a118
I20260812 03:08:03.123453 29380 ts_tablet_manager.cc:1939] T 3e3c3e7ec7954b8aad5bbc9c8905a118 P ecbf547cfa88443ba4f9ac8325dffa5b: Deleting consensus metadata
I20260812 03:08:03.123466 29354 raft_consensus.cc:348] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:03.123641 29354 raft_consensus.cc:374] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:03.123692 29354 raft_consensus.cc:729] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0294274610e94b7a84d2883851f97131, State: Initialized, Role: FOLLOWER
I20260812 03:08:03.123847 29354 consensus_queue.cc:260] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:03.124009 29353 tablet_bootstrap.cc:492] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 94ce0df0f78c4548a748fe7394171fa9: No bootstrap required, opened a new log
I20260812 03:08:03.124107 29353 ts_tablet_manager.cc:1397] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 94ce0df0f78c4548a748fe7394171fa9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:08:03.124182 29354 ts_tablet_manager.cc:1428] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 03:08:03.124605 29353 raft_consensus.cc:348] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 94ce0df0f78c4548a748fe7394171fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:03.124733 29353 raft_consensus.cc:374] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 94ce0df0f78c4548a748fe7394171fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:03.124779 29353 raft_consensus.cc:729] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 94ce0df0f78c4548a748fe7394171fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94ce0df0f78c4548a748fe7394171fa9, State: Initialized, Role: FOLLOWER
I20260812 03:08:03.124753 28790 catalog_manager.cc:4841] TS ecbf547cfa88443ba4f9ac8325dffa5b (127.26.250.131:35227): tablet 3e3c3e7ec7954b8aad5bbc9c8905a118 (table test-table [id=9567799769db42eea7945e85c22a09f0]) successfully deleted
I20260812 03:08:03.124904 29353 consensus_queue.cc:260] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 94ce0df0f78c4548a748fe7394171fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:03.125214 29353 ts_tablet_manager.cc:1428] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 94ce0df0f78c4548a748fe7394171fa9: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 03:08:03.127148 29355 tablet_bootstrap.cc:492] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P ecbf547cfa88443ba4f9ac8325dffa5b: No bootstrap required, opened a new log
I20260812 03:08:03.127424 29355 ts_tablet_manager.cc:1397] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P ecbf547cfa88443ba4f9ac8325dffa5b: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20260812 03:08:03.127895 29355 raft_consensus.cc:348] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P ecbf547cfa88443ba4f9ac8325dffa5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:03.127995 29355 raft_consensus.cc:374] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P ecbf547cfa88443ba4f9ac8325dffa5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:03.128017 29355 raft_consensus.cc:729] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P ecbf547cfa88443ba4f9ac8325dffa5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecbf547cfa88443ba4f9ac8325dffa5b, State: Initialized, Role: FOLLOWER
I20260812 03:08:03.128189 29355 consensus_queue.cc:260] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P ecbf547cfa88443ba4f9ac8325dffa5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:03.128448 29355 ts_tablet_manager.cc:1428] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P ecbf547cfa88443ba4f9ac8325dffa5b: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260812 03:08:03.334794 29366 raft_consensus.cc:482] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:08:03.335000 29366 raft_consensus.cc:504] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:03.335521 29366 leader_election.cc:288] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94ce0df0f78c4548a748fe7394171fa9 (127.26.250.129:34813), ecbf547cfa88443ba4f9ac8325dffa5b (127.26.250.131:35227)
I20260812 03:08:03.336076 28980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ce1b4634fdf4d4a8ea59c2c4ce98d51" candidate_uuid: "0294274610e94b7a84d2883851f97131" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94ce0df0f78c4548a748fe7394171fa9" is_pre_election: true
I20260812 03:08:03.336076 29279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ce1b4634fdf4d4a8ea59c2c4ce98d51" candidate_uuid: "0294274610e94b7a84d2883851f97131" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" is_pre_election: true
I20260812 03:08:03.336350 28980 raft_consensus.cc:2454] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 94ce0df0f78c4548a748fe7394171fa9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0294274610e94b7a84d2883851f97131 in term 0.
I20260812 03:08:03.336350 29279 raft_consensus.cc:2454] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P ecbf547cfa88443ba4f9ac8325dffa5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0294274610e94b7a84d2883851f97131 in term 0.
I20260812 03:08:03.336776 29057 leader_election.cc:302] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0294274610e94b7a84d2883851f97131, ecbf547cfa88443ba4f9ac8325dffa5b; no voters:
I20260812 03:08:03.336977 29366 raft_consensus.cc:2781] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:08:03.337042 29366 raft_consensus.cc:482] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:08:03.337096 29366 raft_consensus.cc:3037] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:03.338274 29366 raft_consensus.cc:504] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:03.338730 29366 leader_election.cc:288] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131 [CANDIDATE]: Term 1 election: Requested vote from peers 94ce0df0f78c4548a748fe7394171fa9 (127.26.250.129:34813), ecbf547cfa88443ba4f9ac8325dffa5b (127.26.250.131:35227)
I20260812 03:08:03.339172 29279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ce1b4634fdf4d4a8ea59c2c4ce98d51" candidate_uuid: "0294274610e94b7a84d2883851f97131" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b"
I20260812 03:08:03.339244 28980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ce1b4634fdf4d4a8ea59c2c4ce98d51" candidate_uuid: "0294274610e94b7a84d2883851f97131" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94ce0df0f78c4548a748fe7394171fa9"
I20260812 03:08:03.339309 29279 raft_consensus.cc:3037] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P ecbf547cfa88443ba4f9ac8325dffa5b [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:03.339365 28980 raft_consensus.cc:3037] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 94ce0df0f78c4548a748fe7394171fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:03.340543 29279 raft_consensus.cc:2454] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P ecbf547cfa88443ba4f9ac8325dffa5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0294274610e94b7a84d2883851f97131 in term 1.
I20260812 03:08:03.340543 28980 raft_consensus.cc:2454] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 94ce0df0f78c4548a748fe7394171fa9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0294274610e94b7a84d2883851f97131 in term 1.
I20260812 03:08:03.340963 29058 leader_election.cc:302] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0294274610e94b7a84d2883851f97131, 94ce0df0f78c4548a748fe7394171fa9, ecbf547cfa88443ba4f9ac8325dffa5b; no voters:
I20260812 03:08:03.341178 29366 raft_consensus.cc:2781] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:03.341279 29366 raft_consensus.cc:686] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131 [term 1 LEADER]: Becoming Leader. State: Replica: 0294274610e94b7a84d2883851f97131, State: Running, Role: LEADER
I20260812 03:08:03.341442 29366 consensus_queue.cc:237] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:03.343237 28802 catalog_manager.cc:5495] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0294274610e94b7a84d2883851f97131 (127.26.250.130). New cstate: current_term: 1 leader_uuid: "0294274610e94b7a84d2883851f97131" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } health_report { overall_health: UNKNOWN } } }
W20260812 03:08:03.343408 29330 tablet.cc:2369] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P ecbf547cfa88443ba4f9ac8325dffa5b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:08:03.351569 29181 tablet.cc:2369] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:08:03.354835 29030 tablet.cc:2369] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 94ce0df0f78c4548a748fe7394171fa9: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 03:08:03 dist-test-slave-zpfg krb5kdc[28759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504081, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:03.411459 29391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:03.400693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:45553 (local address 127.0.0.1:56672)
0812 03:08:03.401322 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:03.401337 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:03.401498 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:03.401821 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:03.401825 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:03.402321 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:03.402471 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:03.402479 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.403370 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.403377 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:03.404088 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:03.404100 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.404617 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.404621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:03.404709 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:03.405469 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:03.405483 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:03.407318 (+ 1835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:03.409598 (+ 2280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:03.409609 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:03.409616 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:03.409925 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:03.410343 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:03.410347 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:03.410349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:03.410482 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:03.410926 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:03.410933 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:03.411057 (+ 124us) client_negotiation.cc:769] Sending connection context
0812 03:08:03.411157 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 03:08:03.411300 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":163,"threads_started":1}
I20260812 03:08:03.411574 29392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:03.400841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56672 (local address 127.26.250.190:45553)
0812 03:08:03.401204 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:03.401215 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:03.401432 (+ 217us) server_negotiation.cc:407] Connection header received
0812 03:08:03.401606 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:03.401610 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:03.401656 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:03.401801 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:03.402608 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.403243 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.404228 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.404487 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.404610 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:03.407497 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:03.407501 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:03.407503 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:03.407526 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:03.409452 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:03.410036 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:03.410039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:03.410040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:03.410128 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:03.410593 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:03.410595 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:03.410596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:03.410813 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:03.410894 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:03.411305 (+ 411us) server_negotiation.cc:299] Negotiation successful
0812 03:08:03.411416 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":300,"thread_start_us":95,"threads_started":1}
I20260812 03:08:03.416850 28802 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:56672:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 03:08:03.417788 28802 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:08:03.424065 28958 tablet_service.cc:1467] Processing CreateTablet for tablet 4cc3e2cdff274dcdb4d22355a540d7c2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3601b9d8ebd74707872865078ae323ab]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:08:03.424118 29256 tablet_service.cc:1467] Processing CreateTablet for tablet 4cc3e2cdff274dcdb4d22355a540d7c2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3601b9d8ebd74707872865078ae323ab]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:08:03.424592 28958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cc3e2cdff274dcdb4d22355a540d7c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:03.424605 29256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cc3e2cdff274dcdb4d22355a540d7c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:03.426888 29109 tablet_service.cc:1467] Processing CreateTablet for tablet 4cc3e2cdff274dcdb4d22355a540d7c2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3601b9d8ebd74707872865078ae323ab]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:08:03.427326 29109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cc3e2cdff274dcdb4d22355a540d7c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:03.427706 29355 tablet_bootstrap.cc:492] T 4cc3e2cdff274dcdb4d22355a540d7c2 P ecbf547cfa88443ba4f9ac8325dffa5b: Bootstrap starting.
I20260812 03:08:03.428676 29355 tablet_bootstrap.cc:654] T 4cc3e2cdff274dcdb4d22355a540d7c2 P ecbf547cfa88443ba4f9ac8325dffa5b: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:03.430037 29354 tablet_bootstrap.cc:492] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 0294274610e94b7a84d2883851f97131: Bootstrap starting.
I20260812 03:08:03.430107 29353 tablet_bootstrap.cc:492] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9: Bootstrap starting.
I20260812 03:08:03.430743 29355 tablet_bootstrap.cc:492] T 4cc3e2cdff274dcdb4d22355a540d7c2 P ecbf547cfa88443ba4f9ac8325dffa5b: No bootstrap required, opened a new log
I20260812 03:08:03.430866 29355 ts_tablet_manager.cc:1397] T 4cc3e2cdff274dcdb4d22355a540d7c2 P ecbf547cfa88443ba4f9ac8325dffa5b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260812 03:08:03.431017 29354 tablet_bootstrap.cc:654] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 0294274610e94b7a84d2883851f97131: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:03.431253 29353 tablet_bootstrap.cc:654] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:03.431438 29355 raft_consensus.cc:348] T 4cc3e2cdff274dcdb4d22355a540d7c2 P ecbf547cfa88443ba4f9ac8325dffa5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:03.431591 29355 raft_consensus.cc:374] T 4cc3e2cdff274dcdb4d22355a540d7c2 P ecbf547cfa88443ba4f9ac8325dffa5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:03.431644 29355 raft_consensus.cc:729] T 4cc3e2cdff274dcdb4d22355a540d7c2 P ecbf547cfa88443ba4f9ac8325dffa5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecbf547cfa88443ba4f9ac8325dffa5b, State: Initialized, Role: FOLLOWER
I20260812 03:08:03.431838 29355 consensus_queue.cc:260] T 4cc3e2cdff274dcdb4d22355a540d7c2 P ecbf547cfa88443ba4f9ac8325dffa5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:03.432183 29355 ts_tablet_manager.cc:1428] T 4cc3e2cdff274dcdb4d22355a540d7c2 P ecbf547cfa88443ba4f9ac8325dffa5b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 03:08:03.432118 29361 tablet_replica.cc:439] T 4cc3e2cdff274dcdb4d22355a540d7c2 P ecbf547cfa88443ba4f9ac8325dffa5b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } } }
I20260812 03:08:03.432300 29361 tablet_replica.cc:442] T 4cc3e2cdff274dcdb4d22355a540d7c2 P ecbf547cfa88443ba4f9ac8325dffa5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:08:03.433118 29353 tablet_bootstrap.cc:492] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9: No bootstrap required, opened a new log
I20260812 03:08:03.433223 29354 tablet_bootstrap.cc:492] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 0294274610e94b7a84d2883851f97131: No bootstrap required, opened a new log
I20260812 03:08:03.433226 29353 ts_tablet_manager.cc:1397] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:08:03.433295 29354 ts_tablet_manager.cc:1397] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 0294274610e94b7a84d2883851f97131: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:08:03.433665 29354 raft_consensus.cc:348] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 0294274610e94b7a84d2883851f97131 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:03.433665 29353 raft_consensus.cc:348] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:03.433784 29354 raft_consensus.cc:374] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 0294274610e94b7a84d2883851f97131 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:03.433784 29353 raft_consensus.cc:374] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:03.433818 29354 raft_consensus.cc:729] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 0294274610e94b7a84d2883851f97131 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0294274610e94b7a84d2883851f97131, State: Initialized, Role: FOLLOWER
I20260812 03:08:03.433818 29353 raft_consensus.cc:729] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94ce0df0f78c4548a748fe7394171fa9, State: Initialized, Role: FOLLOWER
I20260812 03:08:03.433962 29353 consensus_queue.cc:260] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:03.433962 29354 consensus_queue.cc:260] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 0294274610e94b7a84d2883851f97131 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:03.434067 29359 raft_consensus.cc:482] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:08:03.434202 29359 raft_consensus.cc:504] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:03.434248 29366 tablet_replica.cc:439] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 0294274610e94b7a84d2883851f97131: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } } }
I20260812 03:08:03.434455 29366 tablet_replica.cc:442] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 0294274610e94b7a84d2883851f97131: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:08:03.434250 29354 ts_tablet_manager.cc:1428] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 0294274610e94b7a84d2883851f97131: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:08:03.434712 29353 ts_tablet_manager.cc:1428] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:08:03.434749 29359 leader_election.cc:288] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0294274610e94b7a84d2883851f97131 (127.26.250.130:38727), ecbf547cfa88443ba4f9ac8325dffa5b (127.26.250.131:35227)
I20260812 03:08:03.434921 29359 tablet_replica.cc:439] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } } }
I20260812 03:08:03.435060 29359 tablet_replica.cc:442] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:08:03.435148 29279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cc3e2cdff274dcdb4d22355a540d7c2" candidate_uuid: "94ce0df0f78c4548a748fe7394171fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" is_pre_election: true
I20260812 03:08:03.435206 29131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cc3e2cdff274dcdb4d22355a540d7c2" candidate_uuid: "94ce0df0f78c4548a748fe7394171fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0294274610e94b7a84d2883851f97131" is_pre_election: true
I20260812 03:08:03.435299 29279 raft_consensus.cc:2454] T 4cc3e2cdff274dcdb4d22355a540d7c2 P ecbf547cfa88443ba4f9ac8325dffa5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94ce0df0f78c4548a748fe7394171fa9 in term 0.
I20260812 03:08:03.435384 29131 raft_consensus.cc:2454] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 0294274610e94b7a84d2883851f97131 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94ce0df0f78c4548a748fe7394171fa9 in term 0.
I20260812 03:08:03.435899 28901 leader_election.cc:302] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0294274610e94b7a84d2883851f97131, 94ce0df0f78c4548a748fe7394171fa9; no voters:
I20260812 03:08:03.436239 29396 raft_consensus.cc:2781] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:08:03.436321 29396 raft_consensus.cc:482] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:08:03.436384 29396 raft_consensus.cc:3037] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:03.438076 29396 raft_consensus.cc:504] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:03.438524 29396 leader_election.cc:288] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9 [CANDIDATE]: Term 1 election: Requested vote from peers 0294274610e94b7a84d2883851f97131 (127.26.250.130:38727), ecbf547cfa88443ba4f9ac8325dffa5b (127.26.250.131:35227)
I20260812 03:08:03.438910 29131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cc3e2cdff274dcdb4d22355a540d7c2" candidate_uuid: "94ce0df0f78c4548a748fe7394171fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0294274610e94b7a84d2883851f97131"
I20260812 03:08:03.439065 29131 raft_consensus.cc:3037] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 0294274610e94b7a84d2883851f97131 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:03.439039 29279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cc3e2cdff274dcdb4d22355a540d7c2" candidate_uuid: "94ce0df0f78c4548a748fe7394171fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b"
I20260812 03:08:03.439334 29279 raft_consensus.cc:3037] T 4cc3e2cdff274dcdb4d22355a540d7c2 P ecbf547cfa88443ba4f9ac8325dffa5b [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:03.440397 29279 raft_consensus.cc:2454] T 4cc3e2cdff274dcdb4d22355a540d7c2 P ecbf547cfa88443ba4f9ac8325dffa5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94ce0df0f78c4548a748fe7394171fa9 in term 1.
I20260812 03:08:03.440434 29131 raft_consensus.cc:2454] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 0294274610e94b7a84d2883851f97131 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94ce0df0f78c4548a748fe7394171fa9 in term 1.
I20260812 03:08:03.440907 28901 leader_election.cc:302] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0294274610e94b7a84d2883851f97131, 94ce0df0f78c4548a748fe7394171fa9; no voters:
I20260812 03:08:03.441128 29396 raft_consensus.cc:2781] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:03.441247 29396 raft_consensus.cc:686] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9 [term 1 LEADER]: Becoming Leader. State: Replica: 94ce0df0f78c4548a748fe7394171fa9, State: Running, Role: LEADER
I20260812 03:08:03.441529 29396 consensus_queue.cc:237] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } }
I20260812 03:08:03.443573 29359 tablet_replica.cc:439] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9: TxnStatusTablet state changed. Reason: New leader 94ce0df0f78c4548a748fe7394171fa9. Latest consensus state: current_term: 1 leader_uuid: "94ce0df0f78c4548a748fe7394171fa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } } }
I20260812 03:08:03.443696 29359 tablet_replica.cc:442] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:08:03.444080 29398 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 03:08:03.444248 29398 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 03:08:03.445324 28802 catalog_manager.cc:5495] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94ce0df0f78c4548a748fe7394171fa9 (127.26.250.129). New cstate: current_term: 1 leader_uuid: "94ce0df0f78c4548a748fe7394171fa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } health_report { overall_health: UNKNOWN } } }
I20260812 03:08:03.462813 29391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:03.457464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:34813 (local address 127.0.0.1:37968)
0812 03:08:03.457633 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:03.457651 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:03.457792 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:03.458175 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:03.458178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:03.458187 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:08:03.458310 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:03.458316 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.459664 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.459668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:03.460458 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:03.460468 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.461143 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.461146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:03.461292 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:03.461354 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:08:03.462428 (+ 1074us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:08:03.462448 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 03:08:03.462525 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 03:08:03.462644 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 03:08:03.462839 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:03.457616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37968 (local address 127.26.250.129:34813)
0812 03:08:03.457779 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:03.457784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:03.457802 (+ 18us) server_negotiation.cc:407] Connection header received
0812 03:08:03.457972 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:03.457975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:03.458037 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:03.458151 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:08:03.458496 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.459506 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.460598 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.460991 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.461121 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:03.461537 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:08:03.462181 (+ 644us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:08:03.462334 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:03.462566 (+ 232us) server_negotiation.cc:299] Negotiation successful
0812 03:08:03.462667 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:08:03.466815 29279 raft_consensus.cc:1261] T 4cc3e2cdff274dcdb4d22355a540d7c2 P ecbf547cfa88443ba4f9ac8325dffa5b [term 1 FOLLOWER]: Refusing update from remote peer 94ce0df0f78c4548a748fe7394171fa9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:03.466993 29131 raft_consensus.cc:1261] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 0294274610e94b7a84d2883851f97131 [term 1 FOLLOWER]: Refusing update from remote peer 94ce0df0f78c4548a748fe7394171fa9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:03.467406 29396 consensus_queue.cc:1035] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:03.467640 29359 consensus_queue.cc:1035] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:03.469185 29361 tablet_replica.cc:439] T 4cc3e2cdff274dcdb4d22355a540d7c2 P ecbf547cfa88443ba4f9ac8325dffa5b: TxnStatusTablet state changed. Reason: New leader 94ce0df0f78c4548a748fe7394171fa9. Latest consensus state: current_term: 1 leader_uuid: "94ce0df0f78c4548a748fe7394171fa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } } }
I20260812 03:08:03.469339 29361 tablet_replica.cc:442] T 4cc3e2cdff274dcdb4d22355a540d7c2 P ecbf547cfa88443ba4f9ac8325dffa5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:08:03.469913 29366 tablet_replica.cc:439] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 0294274610e94b7a84d2883851f97131: TxnStatusTablet state changed. Reason: New leader 94ce0df0f78c4548a748fe7394171fa9. Latest consensus state: current_term: 1 leader_uuid: "94ce0df0f78c4548a748fe7394171fa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } } }
I20260812 03:08:03.470047 29366 tablet_replica.cc:442] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 0294274610e94b7a84d2883851f97131: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:08:03.471510 29396 tablet_replica.cc:439] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "94ce0df0f78c4548a748fe7394171fa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } } }
I20260812 03:08:03.471635 29396 tablet_replica.cc:442] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:08:03.473088 29369 mvcc.cc:205] Tried to move back new op lower bound from 7317520725873098752 to 7317520725779206144. Current Snapshot: MvccSnapshot[applied={T|T < 7317520725873098752}]
I20260812 03:08:03.473304 29366 tablet_replica.cc:439] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 0294274610e94b7a84d2883851f97131: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "94ce0df0f78c4548a748fe7394171fa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } } }
I20260812 03:08:03.473425 29366 tablet_replica.cc:442] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 0294274610e94b7a84d2883851f97131: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:08:03.474745 29396 tablet_replica.cc:439] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "94ce0df0f78c4548a748fe7394171fa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } } }
I20260812 03:08:03.474889 29396 tablet_replica.cc:442] T 4cc3e2cdff274dcdb4d22355a540d7c2 P 94ce0df0f78c4548a748fe7394171fa9: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:08:03.477216 29361 tablet_replica.cc:439] T 4cc3e2cdff274dcdb4d22355a540d7c2 P ecbf547cfa88443ba4f9ac8325dffa5b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "94ce0df0f78c4548a748fe7394171fa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 } } peers { permanent_uuid: "0294274610e94b7a84d2883851f97131" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 38727 } } peers { permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 } } }
I20260812 03:08:03.477367 29361 tablet_replica.cc:442] T 4cc3e2cdff274dcdb4d22355a540d7c2 P ecbf547cfa88443ba4f9ac8325dffa5b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:08:03.491138 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:03.486797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:42313 (local address 127.26.250.129:34813)
0812 03:08:03.486923 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:03.486927 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:03.487185 (+ 258us) server_negotiation.cc:407] Connection header received
0812 03:08:03.487331 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:03.487334 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:03.487392 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:03.487498 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:08:03.487809 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.488607 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.489614 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.489971 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.490102 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:03.490550 (+ 448us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:08:03.490638 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:08:03.490777 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:03.490920 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 03:08:03.490978 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:08:03.491156 29405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:03.486663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:34813 (local address 127.26.250.130:42313)
0812 03:08:03.487086 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:03.487101 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:03.487219 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:03.487558 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:03.487560 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:03.487569 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:08:03.487677 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:03.487681 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.488738 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.488742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:03.489486 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:03.489497 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.490141 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.490144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:03.490238 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:03.490296 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:08:03.490795 (+ 499us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:08:03.490806 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 03:08:03.490902 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 03:08:03.491006 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":143,"threads_started":1}
I20260812 03:08:03.499931 28980 raft_consensus.cc:1261] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 94ce0df0f78c4548a748fe7394171fa9 [term 1 FOLLOWER]: Refusing update from remote peer 0294274610e94b7a84d2883851f97131: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:03.500363 29279 raft_consensus.cc:1261] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P ecbf547cfa88443ba4f9ac8325dffa5b [term 1 FOLLOWER]: Refusing update from remote peer 0294274610e94b7a84d2883851f97131: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:03.500474 29366 consensus_queue.cc:1035] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94ce0df0f78c4548a748fe7394171fa9" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:03.500854 29375 consensus_queue.cc:1035] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecbf547cfa88443ba4f9ac8325dffa5b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 35227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:03.509218 29370 mvcc.cc:205] Tried to move back new op lower bound from 7317520726011506688 to 7317520725368012800. Current Snapshot: MvccSnapshot[applied={T|T < 7317520726011506688}]
I20260812 03:08:03.531548 29351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:03.527299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:51443 (local address 127.26.250.130:38727)
0812 03:08:03.527466 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:03.527471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:03.527675 (+ 204us) server_negotiation.cc:407] Connection header received
0812 03:08:03.527821 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:03.527824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:03.527880 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:03.527951 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:08:03.528235 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.528833 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.530136 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.530518 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.530597 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:03.531075 (+ 478us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:08:03.531170 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:08:03.531239 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:03.531362 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 03:08:03.531421 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:08:03.531638 29411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:03.527208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:38727 (local address 127.26.250.129:51443)
0812 03:08:03.527583 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:03.527598 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:03.527725 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:03.527986 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:03.527989 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:03.527998 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:08:03.528115 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:03.528121 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.528977 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.528981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:03.529980 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:03.529992 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.530728 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.530732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:03.530827 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:03.530890 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:08:03.531268 (+ 378us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:08:03.531276 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 03:08:03.531350 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 03:08:03.531448 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":134,"threads_started":1}
I20260812 03:08:03.560760 28803 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:03.560956 28803 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:03.563552 28803 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8b7dde5739cb42b7bbf2c7184adc376e: Sending DeleteTablet for 3 replicas of tablet 2ce1b4634fdf4d4a8ea59c2c4ce98d51
I20260812 03:08:03.564254 28958 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ce1b4634fdf4d4a8ea59c2c4ce98d51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:03 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:37958
I20260812 03:08:03.564458 29381 tablet_replica.cc:330] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 94ce0df0f78c4548a748fe7394171fa9: stopping tablet replica
I20260812 03:08:03.564428 29256 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ce1b4634fdf4d4a8ea59c2c4ce98d51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:03 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:49644
I20260812 03:08:03.564576 29381 raft_consensus.cc:2229] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 94ce0df0f78c4548a748fe7394171fa9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:03.564646 29380 tablet_replica.cc:330] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P ecbf547cfa88443ba4f9ac8325dffa5b: stopping tablet replica
I20260812 03:08:03.564709 29381 raft_consensus.cc:2258] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 94ce0df0f78c4548a748fe7394171fa9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:03.564767 29380 raft_consensus.cc:2229] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P ecbf547cfa88443ba4f9ac8325dffa5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:03.564834 29109 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ce1b4634fdf4d4a8ea59c2c4ce98d51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:03 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:50036
I20260812 03:08:03.564893 29380 raft_consensus.cc:2258] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P ecbf547cfa88443ba4f9ac8325dffa5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:03.565061 29379 tablet_replica.cc:330] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131: stopping tablet replica
I20260812 03:08:03.565192 29379 raft_consensus.cc:2229] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:08:03.565418 29379 raft_consensus.cc:2258] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:03.566218 29381 ts_tablet_manager.cc:1905] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 94ce0df0f78c4548a748fe7394171fa9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:03.566280 29379 ts_tablet_manager.cc:1905] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:03.566305 29380 ts_tablet_manager.cc:1905] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P ecbf547cfa88443ba4f9ac8325dffa5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:03.569209 29380 ts_tablet_manager.cc:1918] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P ecbf547cfa88443ba4f9ac8325dffa5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:08:03.569216 29379 ts_tablet_manager.cc:1918] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:08:03.569260 29381 ts_tablet_manager.cc:1918] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 94ce0df0f78c4548a748fe7394171fa9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:08:03.569290 29379 log.cc:1196] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-1/wal/wals/2ce1b4634fdf4d4a8ea59c2c4ce98d51
I20260812 03:08:03.569290 29380 log.cc:1196] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P ecbf547cfa88443ba4f9ac8325dffa5b: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/2ce1b4634fdf4d4a8ea59c2c4ce98d51
I20260812 03:08:03.569334 29381 log.cc:1196] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 94ce0df0f78c4548a748fe7394171fa9: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786504075277362-27626-0/minicluster-data/ts-0/wal/wals/2ce1b4634fdf4d4a8ea59c2c4ce98d51
I20260812 03:08:03.569574 29380 ts_tablet_manager.cc:1939] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P ecbf547cfa88443ba4f9ac8325dffa5b: Deleting consensus metadata
I20260812 03:08:03.569576 29379 ts_tablet_manager.cc:1939] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 0294274610e94b7a84d2883851f97131: Deleting consensus metadata
I20260812 03:08:03.569592 29381 ts_tablet_manager.cc:1939] T 2ce1b4634fdf4d4a8ea59c2c4ce98d51 P 94ce0df0f78c4548a748fe7394171fa9: Deleting consensus metadata
I20260812 03:08:03.570600 28790 catalog_manager.cc:4841] TS ecbf547cfa88443ba4f9ac8325dffa5b (127.26.250.131:35227): tablet 2ce1b4634fdf4d4a8ea59c2c4ce98d51 (table test-table [id=3c6950ef88c946f393eab6294e19c25a]) successfully deleted
I20260812 03:08:03.570833 28790 catalog_manager.cc:4841] TS 0294274610e94b7a84d2883851f97131 (127.26.250.130:38727): tablet 2ce1b4634fdf4d4a8ea59c2c4ce98d51 (table test-table [id=3c6950ef88c946f393eab6294e19c25a]) successfully deleted
I20260812 03:08:03.571017 28791 catalog_manager.cc:4841] TS 94ce0df0f78c4548a748fe7394171fa9 (127.26.250.129:34813): tablet 2ce1b4634fdf4d4a8ea59c2c4ce98d51 (table test-table [id=3c6950ef88c946f393eab6294e19c25a]) successfully deleted
Aug 12 03:08:03 dist-test-slave-zpfg krb5kdc[28759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504082, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.250.129@KRBTEST.COM
I20260812 03:08:03.582386 29349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:03.570289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37972 (local address 127.26.250.129:34813)
0812 03:08:03.570438 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:03.570443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:03.570894 (+ 451us) server_negotiation.cc:407] Connection header received
0812 03:08:03.571096 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:03.571102 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:03.571153 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:03.571254 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:03.572151 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.572695 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.574144 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.574604 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.574699 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:03.578215 (+ 3516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:03.578233 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:03.578253 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:03.578283 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:03.580249 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:03.580820 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:03.580827 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:03.580829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:03.580930 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:03.581254 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:03.581257 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:03.581259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:03.581544 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:03.581788 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:03.582024 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 03:08:03.582165 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260812 03:08:03.583050 29009 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37972
I20260812 03:08:03.593638 29392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:03.585024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56676 (local address 127.26.250.190:45553)
0812 03:08:03.585175 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:03.585179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:03.585253 (+ 74us) server_negotiation.cc:407] Connection header received
0812 03:08:03.585422 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:03.585426 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:03.585482 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:03.585573 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:03.586264 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.587176 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.587990 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:03.588389 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:03.588508 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:03.589977 (+ 1469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:03.589981 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:03.589985 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:03.590018 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:03.591811 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:03.592287 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:03.592291 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:03.592294 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:03.592394 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:03.592650 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:03.592654 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:03.592655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:03.592922 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:03.593065 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:03.593304 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 03:08:03.593454 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260812 03:08:03.594076 28803 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56676
I20260812 03:08:03.595528 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 28884
I20260812 03:08:03.609925 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 29039
I20260812 03:08:03.619700 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 29189
I20260812 03:08:03.628356 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 28774
2026-08-12T03:08:03Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3155 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:08:03 dist-test-slave-zpfg krb5kdc[29420](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:08:03 dist-test-slave-zpfg krb5kdc[29420](info): set up 2 sockets
Aug 12 03:08:03 dist-test-slave-zpfg krb5kdc[29420](info): commencing operation
krb5kdc: starting...
W20260812 03:08:04.738127 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:08:04 dist-test-slave-zpfg krb5kdc[29420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:08:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:08:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:08:04.877815 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:37709
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:37709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:04.994982 29433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:04.995268 29433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:04.995321 29433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:04.999552 29433 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:08:04.999625 29433 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 03:08:04.999643 29433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:04.999662 29433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:04.999679 29433 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:08:04.999694 29433 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:08:05.005499 29433 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:37709
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:37709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:05.006790 29433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:05.007925 29433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:05.017652 29441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:05.017733 29439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:05.017653 29438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:05.017959 29433 server_base.cc:1034] running on GCE node
I20260812 03:08:05.018874 29433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:05.020713 29433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:05.021919 29433 hybrid_clock.cc:648] HybridClock initialized: now 1786504085021849 us; error 97 us; skew 500 ppm
Aug 12 03:08:05 dist-test-slave-zpfg krb5kdc[29420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504085, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:05.025399 29433 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:08:05.026671 29433 webserver.cc:460] Webserver started at http://127.26.250.190:43119/ using document root <none> and password file <none>
I20260812 03:08:05.027356 29433 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:05.027467 29433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:05.027766 29433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:05.030208 29433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "113aae1b1efb4a26876f54a569c6cf69"
format_stamp: "Formatted at 2026-08-12 03:08:05 on dist-test-slave-zpfg"
server_key: "5663aadaaddc7a6f0412cb99224124ad"
server_key_iv: "e214c3c886c55488dd38fc54ce7d84cb"
server_key_version: "encryptionkey@0"
I20260812 03:08:05.030992 29433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "113aae1b1efb4a26876f54a569c6cf69"
format_stamp: "Formatted at 2026-08-12 03:08:05 on dist-test-slave-zpfg"
server_key: "5663aadaaddc7a6f0412cb99224124ad"
server_key_iv: "e214c3c886c55488dd38fc54ce7d84cb"
server_key_version: "encryptionkey@0"
I20260812 03:08:05.035919 29433 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 03:08:05.039286 29449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:05.040956 29433 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:08:05.041265 29433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "113aae1b1efb4a26876f54a569c6cf69"
format_stamp: "Formatted at 2026-08-12 03:08:05 on dist-test-slave-zpfg"
server_key: "5663aadaaddc7a6f0412cb99224124ad"
server_key_iv: "e214c3c886c55488dd38fc54ce7d84cb"
server_key_version: "encryptionkey@0"
I20260812 03:08:05.041440 29433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:05.059383 29433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:05.063418 29433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:05.063668 29433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:05.073442 29433 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:37709
I20260812 03:08:05.073473 29514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:37709 every 8 connection(s)
I20260812 03:08:05.074656 29433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:08:05.075734 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 29433
I20260812 03:08:05.075847 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:08:05.076148 27626 external_mini_cluster.cc:1428] Setting key 7c4980f087f650452e38e1b3086b0e87
I20260812 03:08:05.078115 29515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:05.084935 29515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69: Bootstrap starting.
Aug 12 03:08:05 dist-test-slave-zpfg krb5kdc[29420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:05.087627 29515 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:05.088433 29515 log.cc:824] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:05.090495 29515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69: No bootstrap required, opened a new log
I20260812 03:08:05.092366 29519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:05.077782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36146 (local address 127.26.250.190:37709)
0812 03:08:05.078237 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:05.078258 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:05.078310 (+ 52us) server_negotiation.cc:407] Connection header received
0812 03:08:05.079036 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:05.079052 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:05.079405 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:05.079695 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:05.080639 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.081739 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.082948 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.083496 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.083617 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:05.086751 (+ 3134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:05.086756 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:05.086771 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:05.086800 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:05.089255 (+ 2455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:05.089809 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:05.089814 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:05.089819 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:05.089938 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:05.090240 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:05.090245 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:05.090246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:05.090786 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:05.090992 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:05.091317 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 03:08:05.091623 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":161,"threads_started":1}
I20260812 03:08:05.093717 29515 raft_consensus.cc:348] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "113aae1b1efb4a26876f54a569c6cf69" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 37709 } }
I20260812 03:08:05.093963 29515 raft_consensus.cc:374] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:05.094049 29515 raft_consensus.cc:729] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 113aae1b1efb4a26876f54a569c6cf69, State: Initialized, Role: FOLLOWER
I20260812 03:08:05.094590 29515 consensus_queue.cc:260] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "113aae1b1efb4a26876f54a569c6cf69" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 37709 } }
I20260812 03:08:05.094759 29515 raft_consensus.cc:388] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:08:05.094826 29515 raft_consensus.cc:482] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:08:05.094902 29515 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:05.095829 29515 raft_consensus.cc:504] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "113aae1b1efb4a26876f54a569c6cf69" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 37709 } }
I20260812 03:08:05.096174 29515 leader_election.cc:302] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 113aae1b1efb4a26876f54a569c6cf69; no voters:
I20260812 03:08:05.096468 29515 leader_election.cc:288] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:08:05.096635 29521 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:05.096894 29521 raft_consensus.cc:686] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69 [term 1 LEADER]: Becoming Leader. State: Replica: 113aae1b1efb4a26876f54a569c6cf69, State: Running, Role: LEADER
I20260812 03:08:05.097245 29521 consensus_queue.cc:237] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "113aae1b1efb4a26876f54a569c6cf69" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 37709 } }
I20260812 03:08:05.097839 29515 sys_catalog.cc:564] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:08:05.098532 29523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 113aae1b1efb4a26876f54a569c6cf69. Latest consensus state: current_term: 1 leader_uuid: "113aae1b1efb4a26876f54a569c6cf69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "113aae1b1efb4a26876f54a569c6cf69" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 37709 } } }
I20260812 03:08:05.098672 29523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69 [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:05.099042 29522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "113aae1b1efb4a26876f54a569c6cf69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "113aae1b1efb4a26876f54a569c6cf69" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 37709 } } }
I20260812 03:08:05.099154 29522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69 [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:05.099390 29530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:08:05.102684 29530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:08:05.108107 29530 catalog_manager.cc:1347] Generated new cluster ID: c631aa5e9817435d9bdf2fdba9f5a948
I20260812 03:08:05.108186 29530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:08:05.144955 29530 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:08:05.146101 29530 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:08:05.151365 29530 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69: Generated new TSK 0
I20260812 03:08:05.152257 29530 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.250.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
I20260812 03:08:05.186941 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:0
--local_ip_for_outbound_sockets=127.26.250.129
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:37709
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:05.340262 29543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:05.340716 29543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:05.340808 29543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:05.349139 29543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:05.349257 29543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:05.349345 29543 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 03:08:05.349386 29543 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:08:05.349463 29543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:08:05.360090 29543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.250.190:37709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:05.361673 29543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:05.363436 29543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:05.376524 29548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:05.376519 29549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:05.376591 29552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:05.377638 29543 server_base.cc:1034] running on GCE node
I20260812 03:08:05.378166 29543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:05.378973 29543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:05.380187 29543 hybrid_clock.cc:648] HybridClock initialized: now 1786504085380153 us; error 51 us; skew 500 ppm
Aug 12 03:08:05 dist-test-slave-zpfg krb5kdc[29420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504085, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:05.388589 29543 init.cc:377] Logged in from keytab as kudu/127.26.250.129@KRBTEST.COM (short username kudu)
I20260812 03:08:05.389776 29543 webserver.cc:460] Webserver started at http://127.26.250.129:36927/ using document root <none> and password file <none>
I20260812 03:08:05.390602 29543 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:05.390724 29543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:05.391003 29543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:05.394150 29543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-0/data/instance:
uuid: "7a55e25464f6460f906b4a86a75baa9c"
format_stamp: "Formatted at 2026-08-12 03:08:05 on dist-test-slave-zpfg"
server_key: "338d6f28b2cee22b8ecfc70c2858fa7a"
server_key_iv: "f40befafbdb53a2a7c87b8a6f9a3373a"
server_key_version: "encryptionkey@0"
I20260812 03:08:05.394977 29543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance:
uuid: "7a55e25464f6460f906b4a86a75baa9c"
format_stamp: "Formatted at 2026-08-12 03:08:05 on dist-test-slave-zpfg"
server_key: "338d6f28b2cee22b8ecfc70c2858fa7a"
server_key_iv: "f40befafbdb53a2a7c87b8a6f9a3373a"
server_key_version: "encryptionkey@0"
I20260812 03:08:05.400692 29543 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 03:08:05.403519 29559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:05.404932 29543 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:08:05.405085 29543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "7a55e25464f6460f906b4a86a75baa9c"
format_stamp: "Formatted at 2026-08-12 03:08:05 on dist-test-slave-zpfg"
server_key: "338d6f28b2cee22b8ecfc70c2858fa7a"
server_key_iv: "f40befafbdb53a2a7c87b8a6f9a3373a"
server_key_version: "encryptionkey@0"
I20260812 03:08:05.405230 29543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:05.421576 29543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:05.425374 29543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:05.425623 29543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:05.427456 29543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:05.427538 29543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:05.427623 29543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:05.427683 29543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:08:05 dist-test-slave-zpfg krb5kdc[29420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504085, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:05.440371 29543 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:44973
I20260812 03:08:05.440485 29680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:44973 every 8 connection(s)
I20260812 03:08:05.441697 29543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:08:05.443275 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 29543
I20260812 03:08:05.443377 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance
I20260812 03:08:05.443599 27626 external_mini_cluster.cc:1428] Setting key 19a7450298e4c801a4e5ed260272d050
I20260812 03:08:05.444190 29519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:05.428288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:36751 (local address 127.26.250.190:37709)
0812 03:08:05.428472 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:05.428480 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:05.429803 (+ 1323us) server_negotiation.cc:407] Connection header received
0812 03:08:05.431057 (+ 1254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:05.431060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:05.431126 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:05.431266 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:05.433146 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.434024 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.436367 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.436613 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.436702 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:05.439858 (+ 3156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:05.439862 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:05.439865 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:05.439898 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:05.441744 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:05.442459 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:05.442463 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:05.442465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:05.442562 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:05.442939 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:05.442942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:05.442944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:05.443172 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:05.443307 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:05.443857 (+ 550us) server_negotiation.cc:299] Negotiation successful
0812 03:08:05.444009 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 03:08:05.444980 29568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:05.429137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:37709 (local address 127.26.250.129:36751)
0812 03:08:05.429626 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:05.429681 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:05.430824 (+ 1143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:05.431521 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:05.431533 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:05.432085 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:05.432934 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:05.432963 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.434212 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.434216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:05.436024 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:05.436041 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.436810 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.436816 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:05.437126 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:05.438063 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:05.438084 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:05.439674 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:05.441889 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:05.441895 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:05.441910 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:05.442207 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:05.442650 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:05.442655 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:05.442657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:05.442842 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:05.443308 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:05.443316 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:05.443579 (+ 263us) client_negotiation.cc:769] Sending connection context
0812 03:08:05.443794 (+ 215us) client_negotiation.cc:240] Negotiation successful
0812 03:08:05.444139 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":156,"threads_started":1}
I20260812 03:08:05.446507 29681 heartbeater.cc:347] Connected to a master server at 127.26.250.190:37709
I20260812 03:08:05.446662 29681 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:05.447268 29681 heartbeater.cc:511] Master 127.26.250.190:37709 requested a full tablet report, sending...
I20260812 03:08:05.449183 29467 ts_manager.cc:194] Registered new tserver with Master: 7a55e25464f6460f906b4a86a75baa9c (127.26.250.129:44973)
I20260812 03:08:05.450502 29467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.129@KRBTEST.COM'} at 127.26.250.129:36751
WARNING: no policy specified for kudu/127.26.250.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.130@KRBTEST.COM" created.
I20260812 03:08:05.459329 29688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:05.449116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:37709 (local address 127.26.250.129:34361)
0812 03:08:05.450592 (+ 1476us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:05.450612 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:05.450779 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:05.451078 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:05.451084 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:05.451352 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:05.451499 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:05.451507 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.452558 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.452564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:05.453574 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:05.453589 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.454095 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.454099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:05.454205 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:05.454803 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:05.454821 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:05.455292 (+ 471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:05.457446 (+ 2154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:05.457449 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:05.457450 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:05.457653 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:05.457950 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:05.457952 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:05.457954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:05.458034 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:05.458619 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:05.458621 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:05.458710 (+ 89us) client_negotiation.cc:769] Sending connection context
0812 03:08:05.458855 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 03:08:05.459048 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1352,"thread_start_us":114,"threads_started":1}
I20260812 03:08:05.459372 29519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:05.449216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:34361 (local address 127.26.250.190:37709)
0812 03:08:05.449472 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:05.449479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:05.450694 (+ 1215us) server_negotiation.cc:407] Connection header received
0812 03:08:05.450895 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:05.450898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:05.450943 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:05.451032 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:05.451628 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.452396 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.453730 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.453960 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.454040 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:05.455459 (+ 1419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:05.455462 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:05.455464 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:05.455489 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:05.457329 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:05.457745 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:05.457748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:05.457749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:05.457829 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:05.458208 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:05.458223 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:05.458227 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:05.458511 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:05.458642 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:05.458954 (+ 312us) server_negotiation.cc:299] Negotiation successful
0812 03:08:05.459114 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
I20260812 03:08:05.478518 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:0
--local_ip_for_outbound_sockets=127.26.250.130
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:37709
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:05.599485 29691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:05.599778 29691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:05.599855 29691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:05.604069 29691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:05.604142 29691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:05.604187 29691 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 03:08:05.604202 29691 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:08:05.604242 29691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:08:05.609956 29691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.250.190:37709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:05.611284 29691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:05.612425 29691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:05.623310 29700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:05.623399 29697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:05.623414 29698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:05.623579 29691 server_base.cc:1034] running on GCE node
I20260812 03:08:05.624199 29691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:05.624969 29691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:05.626185 29691 hybrid_clock.cc:648] HybridClock initialized: now 1786504085626153 us; error 50 us; skew 500 ppm
Aug 12 03:08:05 dist-test-slave-zpfg krb5kdc[29420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504085, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:05.629518 29691 init.cc:377] Logged in from keytab as kudu/127.26.250.130@KRBTEST.COM (short username kudu)
I20260812 03:08:05.630610 29691 webserver.cc:460] Webserver started at http://127.26.250.130:42861/ using document root <none> and password file <none>
I20260812 03:08:05.631254 29691 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:05.631361 29691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:05.631625 29691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:05.633939 29691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-1/data/instance:
uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2"
format_stamp: "Formatted at 2026-08-12 03:08:05 on dist-test-slave-zpfg"
server_key: "2221b40f4c5e844890eb545edb27af86"
server_key_iv: "61c31417e82379c2e5d117489854b31f"
server_key_version: "encryptionkey@0"
I20260812 03:08:05.634656 29691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance:
uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2"
format_stamp: "Formatted at 2026-08-12 03:08:05 on dist-test-slave-zpfg"
server_key: "2221b40f4c5e844890eb545edb27af86"
server_key_iv: "61c31417e82379c2e5d117489854b31f"
server_key_version: "encryptionkey@0"
I20260812 03:08:05.639123 29691 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:08:05.641791 29707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:05.643180 29691 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:08:05.643347 29691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2"
format_stamp: "Formatted at 2026-08-12 03:08:05 on dist-test-slave-zpfg"
server_key: "2221b40f4c5e844890eb545edb27af86"
server_key_iv: "61c31417e82379c2e5d117489854b31f"
server_key_version: "encryptionkey@0"
I20260812 03:08:05.643503 29691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:05.664506 29691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:05.668653 29691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:05.668901 29691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:05.670606 29691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:05.670689 29691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:05.670768 29691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:05.670825 29691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:08:05 dist-test-slave-zpfg krb5kdc[29420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504085, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:05.683962 29691 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:32889
I20260812 03:08:05.684046 29830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:32889 every 8 connection(s)
I20260812 03:08:05.684995 29691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:08:05.686450 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 29691
I20260812 03:08:05.686539 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance
I20260812 03:08:05.686885 27626 external_mini_cluster.cc:1428] Setting key 080b9e256674ae62bac17e74f10d85ac
I20260812 03:08:05.688578 29519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:05.671945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:47897 (local address 127.26.250.190:37709)
0812 03:08:05.672092 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:05.672096 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:05.673437 (+ 1341us) server_negotiation.cc:407] Connection header received
0812 03:08:05.674889 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:05.674893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:05.674959 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:05.675082 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:05.676709 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.677634 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.679853 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.680278 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.680384 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:05.684177 (+ 3793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:05.684185 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:05.684188 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:05.684218 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:05.685979 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:05.686704 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:05.686708 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:05.686710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:05.686809 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:05.687238 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:05.687241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:05.687243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:05.687465 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:05.687588 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:05.688265 (+ 677us) server_negotiation.cc:299] Negotiation successful
0812 03:08:05.688418 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 03:08:05.689563 29717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:05.672564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:37709 (local address 127.26.250.130:47897)
0812 03:08:05.673254 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:05.673308 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:05.674640 (+ 1332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:05.675290 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:05.675297 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:05.675736 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:05.676528 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:05.676546 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.677822 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.677827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:05.679673 (+ 1846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:05.679689 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.680472 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.680477 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:05.680745 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:05.681875 (+ 1130us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:05.681905 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:05.683926 (+ 2021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:05.686106 (+ 2180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:05.686112 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:05.686128 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:05.686576 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:05.686910 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:05.686913 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:05.686914 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:05.687118 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:05.687585 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:05.687591 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:05.687947 (+ 356us) client_negotiation.cc:769] Sending connection context
0812 03:08:05.688164 (+ 217us) client_negotiation.cc:240] Negotiation successful
0812 03:08:05.688561 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":172,"threads_started":1}
I20260812 03:08:05.691434 29831 heartbeater.cc:347] Connected to a master server at 127.26.250.190:37709
I20260812 03:08:05.691617 29831 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:05.692165 29831 heartbeater.cc:511] Master 127.26.250.190:37709 requested a full tablet report, sending...
I20260812 03:08:05.693357 29468 ts_manager.cc:194] Registered new tserver with Master: 5cfc8a046bde4bfc8649dd83b66bf0f2 (127.26.250.130:32889)
I20260812 03:08:05.694118 29468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.130@KRBTEST.COM'} at 127.26.250.130:47897
WARNING: no policy specified for kudu/127.26.250.131@KRBTEST.COM; defaulting to no policy
I20260812 03:08:05.701869 29519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:05.694000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:47919 (local address 127.26.250.190:37709)
0812 03:08:05.694123 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:05.694128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:05.694421 (+ 293us) server_negotiation.cc:407] Connection header received
0812 03:08:05.694577 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:05.694580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:05.694623 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:05.694723 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:05.695227 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.695746 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.696667 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.696899 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.697008 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:05.698364 (+ 1356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:05.698369 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:05.698371 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:05.698405 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:05.700018 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:05.700543 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:05.700546 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:05.700548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:05.700624 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:05.700926 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:05.700930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:05.700932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:05.701161 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:05.701275 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:05.701558 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 03:08:05.701687 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 03:08:05.701893 29839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:05.694011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:37709 (local address 127.26.250.130:47919)
0812 03:08:05.694321 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:05.694361 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:05.694467 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:05.694733 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:05.694736 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:05.694969 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:05.695110 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:05.695116 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.695883 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.695886 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:05.696533 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:05.696542 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.697017 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.697021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:05.697114 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:05.697767 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:05.697778 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:05.698110 (+ 332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:05.700166 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:05.700171 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:05.700173 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:05.700415 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:05.700718 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:05.700722 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:05.700724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:05.700815 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:05.701265 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:05.701268 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:05.701378 (+ 110us) client_negotiation.cc:769] Sending connection context
0812 03:08:05.701491 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 03:08:05.701691 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":1,"thread_start_us":110,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
I20260812 03:08:05.721866 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:0
--local_ip_for_outbound_sockets=127.26.250.131
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:37709
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:05.854983 29841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:05.855474 29841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:05.855616 29841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:05.860322 29841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:05.860440 29841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:05.860531 29841 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 03:08:05.860584 29841 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:08:05.860656 29841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:08:05.866902 29841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.250.190:37709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:05.868218 29841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:05.869412 29841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:05.880859 29848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:05.880879 29851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:05.881033 29847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:05.881112 29841 server_base.cc:1034] running on GCE node
I20260812 03:08:05.881817 29841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:05.882676 29841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:05.883921 29841 hybrid_clock.cc:648] HybridClock initialized: now 1786504085883881 us; error 57 us; skew 500 ppm
Aug 12 03:08:05 dist-test-slave-zpfg krb5kdc[29420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504085, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:05.887162 29841 init.cc:377] Logged in from keytab as kudu/127.26.250.131@KRBTEST.COM (short username kudu)
I20260812 03:08:05.888141 29841 webserver.cc:460] Webserver started at http://127.26.250.131:41463/ using document root <none> and password file <none>
I20260812 03:08:05.888738 29841 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:05.888813 29841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:05.889003 29841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:05.891543 29841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-2/data/instance:
uuid: "3fae49eb096c4723af74c0400a310adb"
format_stamp: "Formatted at 2026-08-12 03:08:05 on dist-test-slave-zpfg"
server_key: "f06432cf578795c8d8099b1a7924aa21"
server_key_iv: "6520b3ae186d3265d56ac96d4c18b989"
server_key_version: "encryptionkey@0"
I20260812 03:08:05.892251 29841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance:
uuid: "3fae49eb096c4723af74c0400a310adb"
format_stamp: "Formatted at 2026-08-12 03:08:05 on dist-test-slave-zpfg"
server_key: "f06432cf578795c8d8099b1a7924aa21"
server_key_iv: "6520b3ae186d3265d56ac96d4c18b989"
server_key_version: "encryptionkey@0"
I20260812 03:08:05.896813 29841 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:08:05.900410 29859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:05.901863 29841 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:08:05.902014 29841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "3fae49eb096c4723af74c0400a310adb"
format_stamp: "Formatted at 2026-08-12 03:08:05 on dist-test-slave-zpfg"
server_key: "f06432cf578795c8d8099b1a7924aa21"
server_key_iv: "6520b3ae186d3265d56ac96d4c18b989"
server_key_version: "encryptionkey@0"
I20260812 03:08:05.902153 29841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:05.916472 29841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:05.920241 29841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:05.920485 29841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:05.922070 29841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:05.922122 29841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:05.922164 29841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:05.922226 29841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:08:05 dist-test-slave-zpfg krb5kdc[29420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504085, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:05.936172 29841 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:38449
I20260812 03:08:05.936146 29985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:38449 every 8 connection(s)
I20260812 03:08:05.937228 29841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:08:05.939914 29519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:05.923270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:43677 (local address 127.26.250.190:37709)
0812 03:08:05.923459 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:05.923466 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:05.924849 (+ 1383us) server_negotiation.cc:407] Connection header received
0812 03:08:05.926623 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:05.926627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:05.926710 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:05.926809 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:05.928969 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.929695 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.931419 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.931682 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.931829 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:05.935302 (+ 3473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:05.935307 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:05.935310 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:05.935350 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:05.937212 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:05.937961 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:05.937967 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:05.937969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:05.938080 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:05.938622 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:05.938625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:05.938627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:05.938865 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:05.938956 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:05.939629 (+ 673us) server_negotiation.cc:299] Negotiation successful
0812 03:08:05.939746 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:08:05.940559 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 29841
I20260812 03:08:05.940732 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance
I20260812 03:08:05.941007 29868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:05.923886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:37709 (local address 127.26.250.131:43677)
0812 03:08:05.924629 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:05.924678 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:05.926290 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:05.927127 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:05.927139 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:05.927825 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:05.928730 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:05.928771 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.929889 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.929892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:05.931240 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:05.931253 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.931905 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.931910 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:05.932255 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:05.933061 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:05.933080 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:05.935088 (+ 2008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:05.937358 (+ 2270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:05.937365 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:05.937429 (+ 64us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:05.937801 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:05.938253 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:05.938267 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:05.938270 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:05.938499 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:05.938973 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:05.938978 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:05.939232 (+ 254us) client_negotiation.cc:769] Sending connection context
0812 03:08:05.939535 (+ 303us) client_negotiation.cc:240] Negotiation successful
0812 03:08:05.939832 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":149,"threads_started":1}
I20260812 03:08:05.941075 27626 external_mini_cluster.cc:1428] Setting key da4e18e57dadbfe2f223b130530e800b
I20260812 03:08:05.942677 29986 heartbeater.cc:347] Connected to a master server at 127.26.250.190:37709
I20260812 03:08:05.942850 29986 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:05.943368 29986 heartbeater.cc:511] Master 127.26.250.190:37709 requested a full tablet report, sending...
I20260812 03:08:05.944386 29466 ts_manager.cc:194] Registered new tserver with Master: 3fae49eb096c4723af74c0400a310adb (127.26.250.131:38449)
I20260812 03:08:05.945150 29466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.131@KRBTEST.COM'} at 127.26.250.131:43677
I20260812 03:08:05.953419 29519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:05.945052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:38959 (local address 127.26.250.190:37709)
0812 03:08:05.945202 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:05.945207 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:05.945499 (+ 292us) server_negotiation.cc:407] Connection header received
0812 03:08:05.945688 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:05.945691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:05.945760 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:05.945839 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:05.946451 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.947216 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.948224 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.948501 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.948592 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:05.949982 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:05.949986 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:05.949988 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:05.950011 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:05.951710 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:05.952163 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:05.952166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:05.952168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:05.952250 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:05.952590 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:05.952593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:05.952594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:05.952772 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:05.952865 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:05.953146 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 03:08:05.953253 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 03:08:05.953471 29992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:05.944991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:37709 (local address 127.26.250.131:38959)
0812 03:08:05.945420 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:05.945439 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:05.945574 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:05.945856 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:05.945858 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:05.946127 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:05.946292 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:05.946306 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.947371 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.947374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:05.948112 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:05.948121 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.948676 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.948679 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:05.948764 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:05.949449 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:05.949461 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:05.949815 (+ 354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:05.951823 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:05.951826 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:05.951827 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:05.952068 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:05.952404 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:05.952407 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:05.952408 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:05.952501 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:05.952889 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:05.952891 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:05.952981 (+ 90us) client_negotiation.cc:769] Sending connection context
0812 03:08:05.953082 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 03:08:05.953316 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":170,"threads_started":1}
I20260812 03:08:05.955669 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 03:08:05 dist-test-slave-zpfg krb5kdc[29420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:08:05 dist-test-slave-zpfg krb5kdc[29420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:05.986534 29519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:05.977095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36162 (local address 127.26.250.190:37709)
0812 03:08:05.977323 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:05.977327 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:05.977344 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:08:05.977389 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:05.977391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:05.977434 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:05.977509 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:05.978499 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.979045 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.979880 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:05.980090 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:05.980169 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:05.982852 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:05.982855 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:05.982857 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:05.982883 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:05.984565 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:05.985021 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:05.985024 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:05.985026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:05.985117 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:05.985477 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:05.985479 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:05.985481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:05.985693 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:05.985811 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:05.986053 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 03:08:05.986308 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 03:08:05.990039 29466 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36162:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:08:05.993690 29466 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:08:06.011006 30007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:06.004833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:32889 (local address 127.0.0.1:38002)
0812 03:08:06.005438 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:06.005452 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:06.005536 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:06.006214 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:06.006217 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:06.006235 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:06.006385 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:06.006395 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.007553 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.007557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:06.009260 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:06.009271 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.010382 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.010386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:06.010518 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:06.010562 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 03:08:06.010698 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 03:08:06.010835 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"mutex_wait_us":38,"thread_start_us":100,"threads_started":1}
I20260812 03:08:06.011019 30009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:06.005297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38002 (local address 127.26.250.130:32889)
0812 03:08:06.005744 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:06.005757 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:06.005780 (+ 23us) server_negotiation.cc:407] Connection header received
0812 03:08:06.005912 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:06.005916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:06.006053 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:06.006237 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:06.006523 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.007433 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.009420 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.010212 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.010476 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:06.010651 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:06.010726 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 03:08:06.010829 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":121,"threads_started":1}
I20260812 03:08:06.011288 30006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:06.004440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:44973 (local address 127.0.0.1:46376)
0812 03:08:06.004948 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:06.004964 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:06.005125 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:06.006482 (+ 1357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:06.006485 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:06.006512 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:06.006630 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:06.006637 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.008275 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.008279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:06.009609 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:06.009621 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.010819 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.010823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:06.010929 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:06.010977 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 03:08:06.011042 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 03:08:06.011119 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":20,"spinlock_wait_cycles":12416,"thread_start_us":104,"threads_started":1}
I20260812 03:08:06.011792 30008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:06.005083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46376 (local address 127.26.250.129:44973)
0812 03:08:06.005950 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:06.005959 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:06.005984 (+ 25us) server_negotiation.cc:407] Connection header received
0812 03:08:06.006062 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:06.006067 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:06.006283 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:06.006831 (+ 548us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:06.006977 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.008140 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.009745 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.010668 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.011024 (+ 356us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:06.011183 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:06.011234 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 03:08:06.011657 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":744,"thread_start_us":92,"threads_started":1}
I20260812 03:08:06.012511 30004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:06.003994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:38449 (local address 127.0.0.1:52986)
0812 03:08:06.004829 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:06.004895 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:06.005125 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:06.006012 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:06.006020 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:06.006044 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:06.006203 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:06.006210 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.007959 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.007962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:06.009398 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:06.009419 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.010879 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.010882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:06.012175 (+ 1293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:06.012214 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 03:08:06.012298 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 03:08:06.012372 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"mutex_wait_us":48,"spinlock_wait_cycles":4736,"thread_start_us":139,"threads_started":1}
I20260812 03:08:06.012813 30005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:06.004168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52986 (local address 127.26.250.131:38449)
0812 03:08:06.004775 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:06.004791 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:06.005001 (+ 210us) server_negotiation.cc:407] Connection header received
0812 03:08:06.005336 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:06.005345 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:06.005611 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:06.005771 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:06.006489 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.007799 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.009648 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.010495 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.010668 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:06.010826 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:06.012373 (+ 1547us) server_negotiation.cc:299] Negotiation successful
0812 03:08:06.012495 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":151,"threads_started":1}
I20260812 03:08:06.015007 29760 tablet_service.cc:1467] Processing CreateTablet for tablet 99d33e1d5258424dbe44bac6d677ac10 (DEFAULT_TABLE table=test-table [id=06339273e2a646b1aa7c55159ccd6caf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:06.015403 29612 tablet_service.cc:1467] Processing CreateTablet for tablet 99d33e1d5258424dbe44bac6d677ac10 (DEFAULT_TABLE table=test-table [id=06339273e2a646b1aa7c55159ccd6caf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:06.015429 29911 tablet_service.cc:1467] Processing CreateTablet for tablet 99d33e1d5258424dbe44bac6d677ac10 (DEFAULT_TABLE table=test-table [id=06339273e2a646b1aa7c55159ccd6caf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:06.016404 29760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99d33e1d5258424dbe44bac6d677ac10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:06.016429 29612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99d33e1d5258424dbe44bac6d677ac10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:06.016597 29911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99d33e1d5258424dbe44bac6d677ac10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:06.025305 30011 tablet_bootstrap.cc:492] T 99d33e1d5258424dbe44bac6d677ac10 P 7a55e25464f6460f906b4a86a75baa9c: Bootstrap starting.
I20260812 03:08:06.025460 30012 tablet_bootstrap.cc:492] T 99d33e1d5258424dbe44bac6d677ac10 P 5cfc8a046bde4bfc8649dd83b66bf0f2: Bootstrap starting.
I20260812 03:08:06.027175 30010 tablet_bootstrap.cc:492] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb: Bootstrap starting.
I20260812 03:08:06.027904 30011 tablet_bootstrap.cc:654] T 99d33e1d5258424dbe44bac6d677ac10 P 7a55e25464f6460f906b4a86a75baa9c: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:06.027904 30012 tablet_bootstrap.cc:654] T 99d33e1d5258424dbe44bac6d677ac10 P 5cfc8a046bde4bfc8649dd83b66bf0f2: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:06.028899 30012 log.cc:824] T 99d33e1d5258424dbe44bac6d677ac10 P 5cfc8a046bde4bfc8649dd83b66bf0f2: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:06.028899 30011 log.cc:824] T 99d33e1d5258424dbe44bac6d677ac10 P 7a55e25464f6460f906b4a86a75baa9c: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:06.029464 30010 tablet_bootstrap.cc:654] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:06.030395 30010 log.cc:824] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:06.031276 30012 tablet_bootstrap.cc:492] T 99d33e1d5258424dbe44bac6d677ac10 P 5cfc8a046bde4bfc8649dd83b66bf0f2: No bootstrap required, opened a new log
I20260812 03:08:06.031500 30012 ts_tablet_manager.cc:1397] T 99d33e1d5258424dbe44bac6d677ac10 P 5cfc8a046bde4bfc8649dd83b66bf0f2: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 03:08:06.031808 30011 tablet_bootstrap.cc:492] T 99d33e1d5258424dbe44bac6d677ac10 P 7a55e25464f6460f906b4a86a75baa9c: No bootstrap required, opened a new log
I20260812 03:08:06.031993 30011 ts_tablet_manager.cc:1397] T 99d33e1d5258424dbe44bac6d677ac10 P 7a55e25464f6460f906b4a86a75baa9c: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260812 03:08:06.032505 30010 tablet_bootstrap.cc:492] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb: No bootstrap required, opened a new log
I20260812 03:08:06.032765 30010 ts_tablet_manager.cc:1397] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:08:06.035940 30012 raft_consensus.cc:348] T 99d33e1d5258424dbe44bac6d677ac10 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } }
I20260812 03:08:06.036294 30012 raft_consensus.cc:374] T 99d33e1d5258424dbe44bac6d677ac10 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:06.036103 30010 raft_consensus.cc:348] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } }
I20260812 03:08:06.036373 30012 raft_consensus.cc:729] T 99d33e1d5258424dbe44bac6d677ac10 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cfc8a046bde4bfc8649dd83b66bf0f2, State: Initialized, Role: FOLLOWER
I20260812 03:08:06.036379 30010 raft_consensus.cc:374] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:06.036430 30010 raft_consensus.cc:729] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fae49eb096c4723af74c0400a310adb, State: Initialized, Role: FOLLOWER
I20260812 03:08:06.036542 30011 raft_consensus.cc:348] T 99d33e1d5258424dbe44bac6d677ac10 P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } }
I20260812 03:08:06.036887 30011 raft_consensus.cc:374] T 99d33e1d5258424dbe44bac6d677ac10 P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:06.036860 30012 consensus_queue.cc:260] T 99d33e1d5258424dbe44bac6d677ac10 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } }
I20260812 03:08:06.036952 30011 raft_consensus.cc:729] T 99d33e1d5258424dbe44bac6d677ac10 P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a55e25464f6460f906b4a86a75baa9c, State: Initialized, Role: FOLLOWER
I20260812 03:08:06.036895 30010 consensus_queue.cc:260] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } }
I20260812 03:08:06.037658 29986 heartbeater.cc:503] Master 127.26.250.190:37709 was elected leader, sending a full tablet report...
I20260812 03:08:06.038010 30010 ts_tablet_manager.cc:1428] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 03:08:06.038161 30012 ts_tablet_manager.cc:1428] T 99d33e1d5258424dbe44bac6d677ac10 P 5cfc8a046bde4bfc8649dd83b66bf0f2: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 03:08:06.038432 30011 consensus_queue.cc:260] T 99d33e1d5258424dbe44bac6d677ac10 P 7a55e25464f6460f906b4a86a75baa9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } }
I20260812 03:08:06.038669 29831 heartbeater.cc:503] Master 127.26.250.190:37709 was elected leader, sending a full tablet report...
I20260812 03:08:06.039129 29681 heartbeater.cc:503] Master 127.26.250.190:37709 was elected leader, sending a full tablet report...
I20260812 03:08:06.039412 30011 ts_tablet_manager.cc:1428] T 99d33e1d5258424dbe44bac6d677ac10 P 7a55e25464f6460f906b4a86a75baa9c: Time spent starting tablet: real 0.007s user 0.006s sys 0.002s
I20260812 03:08:06.133600 30017 raft_consensus.cc:482] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:08:06.133940 30017 raft_consensus.cc:504] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } }
I20260812 03:08:06.135288 30017 leader_election.cc:288] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cfc8a046bde4bfc8649dd83b66bf0f2 (127.26.250.130:32889), 7a55e25464f6460f906b4a86a75baa9c (127.26.250.129:44973)
I20260812 03:08:06.140177 30009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:06.135743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:57315 (local address 127.26.250.130:32889)
0812 03:08:06.135889 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:06.135894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:06.135954 (+ 60us) server_negotiation.cc:407] Connection header received
0812 03:08:06.136161 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:06.136164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:06.136363 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:06.136466 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:06.136805 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.137661 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.138943 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.139560 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.139751 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:06.139831 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:06.139994 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 03:08:06.140058 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 03:08:06.140194 29868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:06.135614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:32889 (local address 127.26.250.131:57315)
0812 03:08:06.135864 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:06.135882 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:06.136005 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:06.136494 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:06.136497 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:06.136514 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:06.136657 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:06.136667 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.137788 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.137792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:06.138796 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:06.138816 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.139696 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.139699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:06.139859 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:06.139887 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 03:08:06.139971 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 03:08:06.140051 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138}
I20260812 03:08:06.140892 29783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99d33e1d5258424dbe44bac6d677ac10" candidate_uuid: "3fae49eb096c4723af74c0400a310adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" is_pre_election: true
I20260812 03:08:06.141076 30008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:06.135724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:53381 (local address 127.26.250.129:44973)
0812 03:08:06.135885 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:06.135891 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:06.136349 (+ 458us) server_negotiation.cc:407] Connection header received
0812 03:08:06.136420 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:06.136449 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:06.136504 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:06.136589 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:06.137457 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.138148 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.139821 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.140488 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.140629 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:06.140680 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:06.140889 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 03:08:06.140967 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 03:08:06.141076 30020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:06.135614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:44973 (local address 127.26.250.131:53381)
0812 03:08:06.136178 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:06.136195 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:06.136333 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:06.136634 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:06.136636 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:06.136658 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:06.137305 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:06.137317 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.138307 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.138310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:06.139670 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:06.139680 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.140641 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.140645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:06.140750 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:06.140767 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 03:08:06.140854 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 03:08:06.140925 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":40,"thread_start_us":158,"threads_started":1}
I20260812 03:08:06.141283 29783 raft_consensus.cc:2454] T 99d33e1d5258424dbe44bac6d677ac10 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fae49eb096c4723af74c0400a310adb in term 0.
I20260812 03:08:06.141747 29634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99d33e1d5258424dbe44bac6d677ac10" candidate_uuid: "3fae49eb096c4723af74c0400a310adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a55e25464f6460f906b4a86a75baa9c" is_pre_election: true
I20260812 03:08:06.141832 29862 leader_election.cc:302] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fae49eb096c4723af74c0400a310adb, 5cfc8a046bde4bfc8649dd83b66bf0f2; no voters:
I20260812 03:08:06.142056 29634 raft_consensus.cc:2454] T 99d33e1d5258424dbe44bac6d677ac10 P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fae49eb096c4723af74c0400a310adb in term 0.
I20260812 03:08:06.142172 30017 raft_consensus.cc:2781] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:08:06.142313 30017 raft_consensus.cc:482] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:08:06.142416 30017 raft_consensus.cc:3037] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:06.143673 30017 raft_consensus.cc:504] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } }
I20260812 03:08:06.144095 30017 leader_election.cc:288] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb [CANDIDATE]: Term 1 election: Requested vote from peers 5cfc8a046bde4bfc8649dd83b66bf0f2 (127.26.250.130:32889), 7a55e25464f6460f906b4a86a75baa9c (127.26.250.129:44973)
I20260812 03:08:06.144524 29783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99d33e1d5258424dbe44bac6d677ac10" candidate_uuid: "3fae49eb096c4723af74c0400a310adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2"
I20260812 03:08:06.144547 29634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99d33e1d5258424dbe44bac6d677ac10" candidate_uuid: "3fae49eb096c4723af74c0400a310adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a55e25464f6460f906b4a86a75baa9c"
I20260812 03:08:06.144680 29634 raft_consensus.cc:3037] T 99d33e1d5258424dbe44bac6d677ac10 P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:06.144670 29783 raft_consensus.cc:3037] T 99d33e1d5258424dbe44bac6d677ac10 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:06.145918 29634 raft_consensus.cc:2454] T 99d33e1d5258424dbe44bac6d677ac10 P 7a55e25464f6460f906b4a86a75baa9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fae49eb096c4723af74c0400a310adb in term 1.
I20260812 03:08:06.145937 29783 raft_consensus.cc:2454] T 99d33e1d5258424dbe44bac6d677ac10 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fae49eb096c4723af74c0400a310adb in term 1.
I20260812 03:08:06.146317 29863 leader_election.cc:302] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fae49eb096c4723af74c0400a310adb, 7a55e25464f6460f906b4a86a75baa9c; no voters:
I20260812 03:08:06.146549 30017 raft_consensus.cc:2781] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:06.146839 30017 raft_consensus.cc:686] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb [term 1 LEADER]: Becoming Leader. State: Replica: 3fae49eb096c4723af74c0400a310adb, State: Running, Role: LEADER
I20260812 03:08:06.147230 30017 consensus_queue.cc:237] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } }
I20260812 03:08:06.151479 29466 catalog_manager.cc:5495] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fae49eb096c4723af74c0400a310adb (127.26.250.131). New cstate: current_term: 1 leader_uuid: "3fae49eb096c4723af74c0400a310adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } health_report { overall_health: UNKNOWN } } }
Aug 12 03:08:06 dist-test-slave-zpfg krb5kdc[29420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504085, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:06.185216 29519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:06.176389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36176 (local address 127.26.250.190:37709)
0812 03:08:06.176528 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:06.176532 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:06.176812 (+ 280us) server_negotiation.cc:407] Connection header received
0812 03:08:06.176986 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:06.176990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:06.177033 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:06.177121 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:06.177791 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.178315 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.179178 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.179402 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.179510 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:06.181973 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:06.181976 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:06.181979 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:06.182001 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:06.183470 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:06.183982 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:06.183984 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:06.183986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:06.184057 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:06.184352 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:06.184354 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:06.184355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:06.184526 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:06.184603 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:06.184938 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 03:08:06.185067 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":57}
I20260812 03:08:06.185220 30029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:06.176365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:37709 (local address 127.0.0.1:36176)
0812 03:08:06.176719 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:06.176732 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:06.176888 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:06.177142 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:06.177145 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:06.177574 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:06.177669 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:06.177675 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.178488 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.178493 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:06.179052 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:06.179060 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.179516 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.179519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:06.179616 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:06.180222 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:06.180234 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:06.181825 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:06.183585 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:06.183595 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:06.183601 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:06.183887 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:06.184146 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:06.184148 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:06.184149 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:06.184237 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:06.184648 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:06.184655 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:06.184779 (+ 124us) client_negotiation.cc:769] Sending connection context
0812 03:08:06.184883 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 03:08:06.185012 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":4,"thread_start_us":136,"threads_started":1}
W20260812 03:08:06.186477 29832 tablet.cc:2369] T 99d33e1d5258424dbe44bac6d677ac10 P 5cfc8a046bde4bfc8649dd83b66bf0f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:08:06.188550 29987 tablet.cc:2369] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:08:06.189456 29466 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:36176:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 03:08:06.190287 29466 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260812 03:08:06.193641 29682 tablet.cc:2369] T 99d33e1d5258424dbe44bac6d677ac10 P 7a55e25464f6460f906b4a86a75baa9c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:08:06.195793 29760 tablet_service.cc:1467] Processing CreateTablet for tablet deea4f509a8f4b2a885358b0da9b0a43 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b29aee07518547408818d7474c98ebe4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:08:06.195894 29612 tablet_service.cc:1467] Processing CreateTablet for tablet deea4f509a8f4b2a885358b0da9b0a43 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b29aee07518547408818d7474c98ebe4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:08:06.195964 29911 tablet_service.cc:1467] Processing CreateTablet for tablet deea4f509a8f4b2a885358b0da9b0a43 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b29aee07518547408818d7474c98ebe4]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:08:06.196586 29760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deea4f509a8f4b2a885358b0da9b0a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:06.196673 29612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deea4f509a8f4b2a885358b0da9b0a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:06.196673 29911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deea4f509a8f4b2a885358b0da9b0a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:06.199857 30012 tablet_bootstrap.cc:492] T deea4f509a8f4b2a885358b0da9b0a43 P 5cfc8a046bde4bfc8649dd83b66bf0f2: Bootstrap starting.
I20260812 03:08:06.200114 30011 tablet_bootstrap.cc:492] T deea4f509a8f4b2a885358b0da9b0a43 P 7a55e25464f6460f906b4a86a75baa9c: Bootstrap starting.
I20260812 03:08:06.200860 30012 tablet_bootstrap.cc:654] T deea4f509a8f4b2a885358b0da9b0a43 P 5cfc8a046bde4bfc8649dd83b66bf0f2: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:06.201087 30011 tablet_bootstrap.cc:654] T deea4f509a8f4b2a885358b0da9b0a43 P 7a55e25464f6460f906b4a86a75baa9c: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:06.201848 30010 tablet_bootstrap.cc:492] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb: Bootstrap starting.
I20260812 03:08:06.202641 30011 tablet_bootstrap.cc:492] T deea4f509a8f4b2a885358b0da9b0a43 P 7a55e25464f6460f906b4a86a75baa9c: No bootstrap required, opened a new log
I20260812 03:08:06.202735 30011 ts_tablet_manager.cc:1397] T deea4f509a8f4b2a885358b0da9b0a43 P 7a55e25464f6460f906b4a86a75baa9c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260812 03:08:06.202947 30012 tablet_bootstrap.cc:492] T deea4f509a8f4b2a885358b0da9b0a43 P 5cfc8a046bde4bfc8649dd83b66bf0f2: No bootstrap required, opened a new log
I20260812 03:08:06.203032 30012 ts_tablet_manager.cc:1397] T deea4f509a8f4b2a885358b0da9b0a43 P 5cfc8a046bde4bfc8649dd83b66bf0f2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:08:06.203148 30010 tablet_bootstrap.cc:654] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:06.203255 30011 raft_consensus.cc:348] T deea4f509a8f4b2a885358b0da9b0a43 P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:06.203402 30011 raft_consensus.cc:374] T deea4f509a8f4b2a885358b0da9b0a43 P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:06.203449 30011 raft_consensus.cc:729] T deea4f509a8f4b2a885358b0da9b0a43 P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a55e25464f6460f906b4a86a75baa9c, State: Initialized, Role: FOLLOWER
I20260812 03:08:06.203449 30012 raft_consensus.cc:348] T deea4f509a8f4b2a885358b0da9b0a43 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:06.203563 30012 raft_consensus.cc:374] T deea4f509a8f4b2a885358b0da9b0a43 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:06.203603 30012 raft_consensus.cc:729] T deea4f509a8f4b2a885358b0da9b0a43 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cfc8a046bde4bfc8649dd83b66bf0f2, State: Initialized, Role: FOLLOWER
I20260812 03:08:06.203680 30011 consensus_queue.cc:260] T deea4f509a8f4b2a885358b0da9b0a43 P 7a55e25464f6460f906b4a86a75baa9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:06.203753 30012 consensus_queue.cc:260] T deea4f509a8f4b2a885358b0da9b0a43 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:06.203965 30019 tablet_replica.cc:439] T deea4f509a8f4b2a885358b0da9b0a43 P 7a55e25464f6460f906b4a86a75baa9c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } } }
I20260812 03:08:06.204030 30018 tablet_replica.cc:439] T deea4f509a8f4b2a885358b0da9b0a43 P 5cfc8a046bde4bfc8649dd83b66bf0f2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } } }
I20260812 03:08:06.204123 30019 tablet_replica.cc:442] T deea4f509a8f4b2a885358b0da9b0a43 P 7a55e25464f6460f906b4a86a75baa9c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:08:06.204149 30018 tablet_replica.cc:442] T deea4f509a8f4b2a885358b0da9b0a43 P 5cfc8a046bde4bfc8649dd83b66bf0f2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:08:06.204392 30012 ts_tablet_manager.cc:1428] T deea4f509a8f4b2a885358b0da9b0a43 P 5cfc8a046bde4bfc8649dd83b66bf0f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:08:06.204759 30011 ts_tablet_manager.cc:1428] T deea4f509a8f4b2a885358b0da9b0a43 P 7a55e25464f6460f906b4a86a75baa9c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 03:08:06.205614 30010 tablet_bootstrap.cc:492] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb: No bootstrap required, opened a new log
I20260812 03:08:06.205704 30010 ts_tablet_manager.cc:1397] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 03:08:06.206271 30010 raft_consensus.cc:348] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:06.206449 30010 raft_consensus.cc:374] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:06.206497 30010 raft_consensus.cc:729] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fae49eb096c4723af74c0400a310adb, State: Initialized, Role: FOLLOWER
I20260812 03:08:06.206647 30010 consensus_queue.cc:260] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:06.206954 30010 ts_tablet_manager.cc:1428] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:08:06.206929 30017 tablet_replica.cc:439] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } } }
I20260812 03:08:06.207067 30017 tablet_replica.cc:442] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:08:06.514727 30017 raft_consensus.cc:482] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:08:06.514952 30017 raft_consensus.cc:504] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:06.515599 30017 leader_election.cc:288] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a55e25464f6460f906b4a86a75baa9c (127.26.250.129:44973), 5cfc8a046bde4bfc8649dd83b66bf0f2 (127.26.250.130:32889)
I20260812 03:08:06.516168 29634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deea4f509a8f4b2a885358b0da9b0a43" candidate_uuid: "3fae49eb096c4723af74c0400a310adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a55e25464f6460f906b4a86a75baa9c" is_pre_election: true
I20260812 03:08:06.516179 29783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deea4f509a8f4b2a885358b0da9b0a43" candidate_uuid: "3fae49eb096c4723af74c0400a310adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" is_pre_election: true
I20260812 03:08:06.516367 29634 raft_consensus.cc:2454] T deea4f509a8f4b2a885358b0da9b0a43 P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fae49eb096c4723af74c0400a310adb in term 0.
I20260812 03:08:06.516471 29783 raft_consensus.cc:2454] T deea4f509a8f4b2a885358b0da9b0a43 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fae49eb096c4723af74c0400a310adb in term 0.
I20260812 03:08:06.516729 29863 leader_election.cc:302] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fae49eb096c4723af74c0400a310adb, 7a55e25464f6460f906b4a86a75baa9c; no voters:
I20260812 03:08:06.517009 30017 raft_consensus.cc:2781] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:08:06.517083 30017 raft_consensus.cc:482] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:08:06.517114 30017 raft_consensus.cc:3037] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:06.518245 30017 raft_consensus.cc:504] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:06.518663 30017 leader_election.cc:288] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb [CANDIDATE]: Term 1 election: Requested vote from peers 7a55e25464f6460f906b4a86a75baa9c (127.26.250.129:44973), 5cfc8a046bde4bfc8649dd83b66bf0f2 (127.26.250.130:32889)
I20260812 03:08:06.519094 29634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deea4f509a8f4b2a885358b0da9b0a43" candidate_uuid: "3fae49eb096c4723af74c0400a310adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a55e25464f6460f906b4a86a75baa9c"
I20260812 03:08:06.519109 29783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deea4f509a8f4b2a885358b0da9b0a43" candidate_uuid: "3fae49eb096c4723af74c0400a310adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2"
I20260812 03:08:06.519254 29634 raft_consensus.cc:3037] T deea4f509a8f4b2a885358b0da9b0a43 P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:06.519266 29783 raft_consensus.cc:3037] T deea4f509a8f4b2a885358b0da9b0a43 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:06.520255 29783 raft_consensus.cc:2454] T deea4f509a8f4b2a885358b0da9b0a43 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fae49eb096c4723af74c0400a310adb in term 1.
I20260812 03:08:06.520259 29634 raft_consensus.cc:2454] T deea4f509a8f4b2a885358b0da9b0a43 P 7a55e25464f6460f906b4a86a75baa9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fae49eb096c4723af74c0400a310adb in term 1.
I20260812 03:08:06.520645 29863 leader_election.cc:302] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fae49eb096c4723af74c0400a310adb, 7a55e25464f6460f906b4a86a75baa9c; no voters:
I20260812 03:08:06.520848 30017 raft_consensus.cc:2781] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:06.520943 30017 raft_consensus.cc:686] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb [term 1 LEADER]: Becoming Leader. State: Replica: 3fae49eb096c4723af74c0400a310adb, State: Running, Role: LEADER
I20260812 03:08:06.521077 30017 consensus_queue.cc:237] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:06.521919 30021 tablet_replica.cc:439] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb: TxnStatusTablet state changed. Reason: New leader 3fae49eb096c4723af74c0400a310adb. Latest consensus state: current_term: 1 leader_uuid: "3fae49eb096c4723af74c0400a310adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } } }
I20260812 03:08:06.522045 30021 tablet_replica.cc:442] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:08:06.522394 30035 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 03:08:06.522673 30035 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 03:08:06.523232 29468 catalog_manager.cc:5495] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fae49eb096c4723af74c0400a310adb (127.26.250.131). New cstate: current_term: 1 leader_uuid: "3fae49eb096c4723af74c0400a310adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260812 03:08:06.576372 30029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:06.571496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:38449 (local address 127.0.0.1:52998)
0812 03:08:06.571683 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:06.571709 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:06.571831 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:06.572386 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:06.572389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:06.572399 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:08:06.572529 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:06.572535 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.573411 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.573416 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:06.574417 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:06.574430 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.575001 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.575005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:06.575115 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:06.575170 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:08:06.575983 (+ 813us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:08:06.576005 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:08:06.576087 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 03:08:06.576230 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 03:08:06.576406 30038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:06.571691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52998 (local address 127.26.250.131:38449)
0812 03:08:06.572021 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:06.572031 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:06.572044 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:08:06.572117 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:06.572119 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:06.572190 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:06.572353 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:08:06.572659 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.573283 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.574565 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.574864 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.574994 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:06.575382 (+ 388us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:08:06.575866 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:08:06.576014 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:06.576099 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 03:08:06.576181 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":92,"threads_started":1}
I20260812 03:08:06.582525 29634 raft_consensus.cc:1261] T deea4f509a8f4b2a885358b0da9b0a43 P 7a55e25464f6460f906b4a86a75baa9c [term 1 FOLLOWER]: Refusing update from remote peer 3fae49eb096c4723af74c0400a310adb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:06.582521 29783 raft_consensus.cc:1261] T deea4f509a8f4b2a885358b0da9b0a43 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 1 FOLLOWER]: Refusing update from remote peer 3fae49eb096c4723af74c0400a310adb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:06.583384 30021 consensus_queue.cc:1035] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:06.583630 30017 consensus_queue.cc:1035] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:06.587615 30019 tablet_replica.cc:439] T deea4f509a8f4b2a885358b0da9b0a43 P 7a55e25464f6460f906b4a86a75baa9c: TxnStatusTablet state changed. Reason: New leader 3fae49eb096c4723af74c0400a310adb. Latest consensus state: current_term: 1 leader_uuid: "3fae49eb096c4723af74c0400a310adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } } }
I20260812 03:08:06.587772 30019 tablet_replica.cc:442] T deea4f509a8f4b2a885358b0da9b0a43 P 7a55e25464f6460f906b4a86a75baa9c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:08:06.587973 30018 tablet_replica.cc:439] T deea4f509a8f4b2a885358b0da9b0a43 P 5cfc8a046bde4bfc8649dd83b66bf0f2: TxnStatusTablet state changed. Reason: New leader 3fae49eb096c4723af74c0400a310adb. Latest consensus state: current_term: 1 leader_uuid: "3fae49eb096c4723af74c0400a310adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } } }
I20260812 03:08:06.588109 30018 tablet_replica.cc:442] T deea4f509a8f4b2a885358b0da9b0a43 P 5cfc8a046bde4bfc8649dd83b66bf0f2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:08:06.592805 30039 mvcc.cc:205] Tried to move back new op lower bound from 7317520738633785344 to 7317520738392395776. Current Snapshot: MvccSnapshot[applied={T|T < 7317520738633785344}]
I20260812 03:08:06.593228 30021 tablet_replica.cc:439] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3fae49eb096c4723af74c0400a310adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } } }
I20260812 03:08:06.593377 30021 tablet_replica.cc:442] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:08:06.593393 30018 tablet_replica.cc:439] T deea4f509a8f4b2a885358b0da9b0a43 P 5cfc8a046bde4bfc8649dd83b66bf0f2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3fae49eb096c4723af74c0400a310adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } } }
I20260812 03:08:06.593513 30018 tablet_replica.cc:442] T deea4f509a8f4b2a885358b0da9b0a43 P 5cfc8a046bde4bfc8649dd83b66bf0f2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:08:06.593520 30021 tablet_replica.cc:439] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3fae49eb096c4723af74c0400a310adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } } }
I20260812 03:08:06.593602 30021 tablet_replica.cc:442] T deea4f509a8f4b2a885358b0da9b0a43 P 3fae49eb096c4723af74c0400a310adb: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:08:06.594362 30040 mvcc.cc:205] Tried to move back new op lower bound from 7317520738633785344 to 7317520738392395776. Current Snapshot: MvccSnapshot[applied={T|T < 7317520738633785344}]
I20260812 03:08:06.594304 30019 tablet_replica.cc:439] T deea4f509a8f4b2a885358b0da9b0a43 P 7a55e25464f6460f906b4a86a75baa9c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3fae49eb096c4723af74c0400a310adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } } }
I20260812 03:08:06.594975 30019 tablet_replica.cc:442] T deea4f509a8f4b2a885358b0da9b0a43 P 7a55e25464f6460f906b4a86a75baa9c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:08:06.597122 30041 mvcc.cc:205] Tried to move back new op lower bound from 7317520738633785344 to 7317520738392395776. Current Snapshot: MvccSnapshot[applied={T|T < 7317520738633785344 or (T in {7317520738633785344})}]
I20260812 03:08:06.606983 30038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:06.602870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53004 (local address 127.26.250.131:38449)
0812 03:08:06.603012 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:06.603017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:06.603177 (+ 160us) server_negotiation.cc:407] Connection header received
0812 03:08:06.603280 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:06.603283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:06.603344 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:06.603439 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:08:06.603695 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.604535 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.605524 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.605877 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.606022 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:06.606490 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:08:06.606583 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:08:06.606658 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:06.606770 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 03:08:06.606832 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 03:08:06.620342 30052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:06.616287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:38449 (local address 127.26.250.131:60479)
0812 03:08:06.616653 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:06.616672 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:06.616832 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:06.617128 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:06.617131 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:06.617138 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:08:06.617248 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:06.617253 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.618118 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.618124 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:06.618895 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:06.618904 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.619406 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.619410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:06.619494 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:06.619537 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:08:06.619892 (+ 355us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:08:06.619900 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 03:08:06.619999 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 03:08:06.620153 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":25,"thread_start_us":102,"threads_started":1}
I20260812 03:08:06.620340 30038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:06.616491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:60479 (local address 127.26.250.131:38449)
0812 03:08:06.616620 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:06.616625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:06.616763 (+ 138us) server_negotiation.cc:407] Connection header received
0812 03:08:06.616944 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:06.616948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:06.617001 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:06.617088 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:08:06.617368 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.617989 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.619024 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:06.619268 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:06.619388 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:06.619706 (+ 318us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:08:06.619788 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:08:06.619888 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:06.620011 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 03:08:06.620136 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":57}
I20260812 03:08:06.628926 29634 raft_consensus.cc:1261] T 99d33e1d5258424dbe44bac6d677ac10 P 7a55e25464f6460f906b4a86a75baa9c [term 1 FOLLOWER]: Refusing update from remote peer 3fae49eb096c4723af74c0400a310adb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:06.629026 29783 raft_consensus.cc:1261] T 99d33e1d5258424dbe44bac6d677ac10 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 1 FOLLOWER]: Refusing update from remote peer 3fae49eb096c4723af74c0400a310adb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:06.629491 30045 consensus_queue.cc:1035] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:06.629657 30021 consensus_queue.cc:1035] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:06.680169 29466 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:06.680436 29466 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:06.683372 29466 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69: Sending DeleteTablet for 3 replicas of tablet 99d33e1d5258424dbe44bac6d677ac10
I20260812 03:08:06.684345 29912 tablet_service.cc:1514] Processing DeleteTablet for tablet 99d33e1d5258424dbe44bac6d677ac10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:06 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:52986
I20260812 03:08:06.684420 29760 tablet_service.cc:1514] Processing DeleteTablet for tablet 99d33e1d5258424dbe44bac6d677ac10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:06 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:38002
I20260812 03:08:06.684882 30057 tablet_replica.cc:330] T 99d33e1d5258424dbe44bac6d677ac10 P 5cfc8a046bde4bfc8649dd83b66bf0f2: stopping tablet replica
I20260812 03:08:06.685187 30057 raft_consensus.cc:2229] T 99d33e1d5258424dbe44bac6d677ac10 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:06.685480 30056 tablet_replica.cc:330] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb: stopping tablet replica
I20260812 03:08:06.685849 30056 raft_consensus.cc:2229] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:08:06.686061 29466 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36162:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 03:08:06.686510 30056 raft_consensus.cc:2258] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 03:08:06.686527 29466 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:08:06.687335 30057 raft_consensus.cc:2258] T 99d33e1d5258424dbe44bac6d677ac10 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:06.688081 30056 ts_tablet_manager.cc:1905] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:06.688655 30057 ts_tablet_manager.cc:1905] T 99d33e1d5258424dbe44bac6d677ac10 P 5cfc8a046bde4bfc8649dd83b66bf0f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:06.691776 30056 ts_tablet_manager.cc:1918] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:08:06.691949 30056 log.cc:1196] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/99d33e1d5258424dbe44bac6d677ac10
I20260812 03:08:06.692260 30057 ts_tablet_manager.cc:1918] T 99d33e1d5258424dbe44bac6d677ac10 P 5cfc8a046bde4bfc8649dd83b66bf0f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:08:06.692373 30057 log.cc:1196] T 99d33e1d5258424dbe44bac6d677ac10 P 5cfc8a046bde4bfc8649dd83b66bf0f2: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-1/wal/wals/99d33e1d5258424dbe44bac6d677ac10
I20260812 03:08:06.692323 30056 ts_tablet_manager.cc:1939] T 99d33e1d5258424dbe44bac6d677ac10 P 3fae49eb096c4723af74c0400a310adb: Deleting consensus metadata
I20260812 03:08:06.692740 30057 ts_tablet_manager.cc:1939] T 99d33e1d5258424dbe44bac6d677ac10 P 5cfc8a046bde4bfc8649dd83b66bf0f2: Deleting consensus metadata
I20260812 03:08:06.693836 29452 catalog_manager.cc:4841] TS 3fae49eb096c4723af74c0400a310adb (127.26.250.131:38449): tablet 99d33e1d5258424dbe44bac6d677ac10 (table test-table [id=06339273e2a646b1aa7c55159ccd6caf]) successfully deleted
I20260812 03:08:06.694161 29452 catalog_manager.cc:4841] TS 5cfc8a046bde4bfc8649dd83b66bf0f2 (127.26.250.130:32889): tablet 99d33e1d5258424dbe44bac6d677ac10 (table test-table [id=06339273e2a646b1aa7c55159ccd6caf]) successfully deleted
I20260812 03:08:06.694504 29760 tablet_service.cc:1467] Processing CreateTablet for tablet 83c9d5fc12064264a4afc721f5c87253 (DEFAULT_TABLE table=test-table [id=84b7d2d2cc4c4758850922efc951a82c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:06.694696 29912 tablet_service.cc:1467] Processing CreateTablet for tablet 83c9d5fc12064264a4afc721f5c87253 (DEFAULT_TABLE table=test-table [id=84b7d2d2cc4c4758850922efc951a82c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:06.694801 29760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83c9d5fc12064264a4afc721f5c87253. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:06.695029 29912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83c9d5fc12064264a4afc721f5c87253. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:06.695257 29611 tablet_service.cc:1467] Processing CreateTablet for tablet 83c9d5fc12064264a4afc721f5c87253 (DEFAULT_TABLE table=test-table [id=84b7d2d2cc4c4758850922efc951a82c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:06.695566 29611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83c9d5fc12064264a4afc721f5c87253. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:06.695533 29612 tablet_service.cc:1514] Processing DeleteTablet for tablet 99d33e1d5258424dbe44bac6d677ac10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:06 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:46376
I20260812 03:08:06.697319 30012 tablet_bootstrap.cc:492] T 83c9d5fc12064264a4afc721f5c87253 P 5cfc8a046bde4bfc8649dd83b66bf0f2: Bootstrap starting.
I20260812 03:08:06.697814 30010 tablet_bootstrap.cc:492] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb: Bootstrap starting.
I20260812 03:08:06.698258 30012 tablet_bootstrap.cc:654] T 83c9d5fc12064264a4afc721f5c87253 P 5cfc8a046bde4bfc8649dd83b66bf0f2: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:06.698323 30011 tablet_bootstrap.cc:492] T 83c9d5fc12064264a4afc721f5c87253 P 7a55e25464f6460f906b4a86a75baa9c: Bootstrap starting.
I20260812 03:08:06.698843 30010 tablet_bootstrap.cc:654] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:06.699335 30011 tablet_bootstrap.cc:654] T 83c9d5fc12064264a4afc721f5c87253 P 7a55e25464f6460f906b4a86a75baa9c: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:06.699970 30012 tablet_bootstrap.cc:492] T 83c9d5fc12064264a4afc721f5c87253 P 5cfc8a046bde4bfc8649dd83b66bf0f2: No bootstrap required, opened a new log
I20260812 03:08:06.700088 30012 ts_tablet_manager.cc:1397] T 83c9d5fc12064264a4afc721f5c87253 P 5cfc8a046bde4bfc8649dd83b66bf0f2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:08:06.700634 30012 raft_consensus.cc:348] T 83c9d5fc12064264a4afc721f5c87253 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:06.700744 30010 tablet_bootstrap.cc:492] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb: No bootstrap required, opened a new log
I20260812 03:08:06.700760 30012 raft_consensus.cc:374] T 83c9d5fc12064264a4afc721f5c87253 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:06.700793 30012 raft_consensus.cc:729] T 83c9d5fc12064264a4afc721f5c87253 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cfc8a046bde4bfc8649dd83b66bf0f2, State: Initialized, Role: FOLLOWER
I20260812 03:08:06.700816 30010 ts_tablet_manager.cc:1397] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260812 03:08:06.700964 30012 consensus_queue.cc:260] T 83c9d5fc12064264a4afc721f5c87253 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:06.701171 30010 raft_consensus.cc:348] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:06.701251 30012 ts_tablet_manager.cc:1428] T 83c9d5fc12064264a4afc721f5c87253 P 5cfc8a046bde4bfc8649dd83b66bf0f2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:08:06.701268 30010 raft_consensus.cc:374] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:06.701296 30010 raft_consensus.cc:729] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fae49eb096c4723af74c0400a310adb, State: Initialized, Role: FOLLOWER
I20260812 03:08:06.701404 30010 consensus_queue.cc:260] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:06.701648 30010 ts_tablet_manager.cc:1428] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 03:08:06.702620 30058 tablet_replica.cc:330] T 99d33e1d5258424dbe44bac6d677ac10 P 7a55e25464f6460f906b4a86a75baa9c: stopping tablet replica
I20260812 03:08:06.702975 30058 raft_consensus.cc:2229] T 99d33e1d5258424dbe44bac6d677ac10 P 7a55e25464f6460f906b4a86a75baa9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:06.703182 30058 raft_consensus.cc:2258] T 99d33e1d5258424dbe44bac6d677ac10 P 7a55e25464f6460f906b4a86a75baa9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:06.704103 30011 tablet_bootstrap.cc:492] T 83c9d5fc12064264a4afc721f5c87253 P 7a55e25464f6460f906b4a86a75baa9c: No bootstrap required, opened a new log
I20260812 03:08:06.704212 30011 ts_tablet_manager.cc:1397] T 83c9d5fc12064264a4afc721f5c87253 P 7a55e25464f6460f906b4a86a75baa9c: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260812 03:08:06.704638 30011 raft_consensus.cc:348] T 83c9d5fc12064264a4afc721f5c87253 P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:06.704756 30011 raft_consensus.cc:374] T 83c9d5fc12064264a4afc721f5c87253 P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:06.704790 30011 raft_consensus.cc:729] T 83c9d5fc12064264a4afc721f5c87253 P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a55e25464f6460f906b4a86a75baa9c, State: Initialized, Role: FOLLOWER
I20260812 03:08:06.704908 30011 consensus_queue.cc:260] T 83c9d5fc12064264a4afc721f5c87253 P 7a55e25464f6460f906b4a86a75baa9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:06.705192 30011 ts_tablet_manager.cc:1428] T 83c9d5fc12064264a4afc721f5c87253 P 7a55e25464f6460f906b4a86a75baa9c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:08:06.705336 30058 ts_tablet_manager.cc:1905] T 99d33e1d5258424dbe44bac6d677ac10 P 7a55e25464f6460f906b4a86a75baa9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:06.708655 30058 ts_tablet_manager.cc:1918] T 99d33e1d5258424dbe44bac6d677ac10 P 7a55e25464f6460f906b4a86a75baa9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:08:06.708793 30058 log.cc:1196] T 99d33e1d5258424dbe44bac6d677ac10 P 7a55e25464f6460f906b4a86a75baa9c: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-0/wal/wals/99d33e1d5258424dbe44bac6d677ac10
I20260812 03:08:06.709152 30058 ts_tablet_manager.cc:1939] T 99d33e1d5258424dbe44bac6d677ac10 P 7a55e25464f6460f906b4a86a75baa9c: Deleting consensus metadata
I20260812 03:08:06.710551 29453 catalog_manager.cc:4841] TS 7a55e25464f6460f906b4a86a75baa9c (127.26.250.129:44973): tablet 99d33e1d5258424dbe44bac6d677ac10 (table test-table [id=06339273e2a646b1aa7c55159ccd6caf]) successfully deleted
I20260812 03:08:06.782967 30021 raft_consensus.cc:482] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:08:06.783146 30021 raft_consensus.cc:504] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:06.783849 30021 leader_election.cc:288] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7a55e25464f6460f906b4a86a75baa9c (127.26.250.129:44973), 5cfc8a046bde4bfc8649dd83b66bf0f2 (127.26.250.130:32889)
I20260812 03:08:06.784273 29633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83c9d5fc12064264a4afc721f5c87253" candidate_uuid: "3fae49eb096c4723af74c0400a310adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a55e25464f6460f906b4a86a75baa9c" is_pre_election: true
I20260812 03:08:06.784457 29633 raft_consensus.cc:2454] T 83c9d5fc12064264a4afc721f5c87253 P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fae49eb096c4723af74c0400a310adb in term 0.
I20260812 03:08:06.784531 29783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83c9d5fc12064264a4afc721f5c87253" candidate_uuid: "3fae49eb096c4723af74c0400a310adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" is_pre_election: true
I20260812 03:08:06.784703 29783 raft_consensus.cc:2454] T 83c9d5fc12064264a4afc721f5c87253 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fae49eb096c4723af74c0400a310adb in term 0.
I20260812 03:08:06.784845 29863 leader_election.cc:302] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fae49eb096c4723af74c0400a310adb, 7a55e25464f6460f906b4a86a75baa9c; no voters:
I20260812 03:08:06.785133 30021 raft_consensus.cc:2781] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:08:06.785212 30021 raft_consensus.cc:482] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:08:06.785254 30021 raft_consensus.cc:3037] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:06.786597 30021 raft_consensus.cc:504] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:06.787077 30021 leader_election.cc:288] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb [CANDIDATE]: Term 1 election: Requested vote from peers 7a55e25464f6460f906b4a86a75baa9c (127.26.250.129:44973), 5cfc8a046bde4bfc8649dd83b66bf0f2 (127.26.250.130:32889)
I20260812 03:08:06.787580 29783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83c9d5fc12064264a4afc721f5c87253" candidate_uuid: "3fae49eb096c4723af74c0400a310adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2"
I20260812 03:08:06.787721 29783 raft_consensus.cc:3037] T 83c9d5fc12064264a4afc721f5c87253 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:06.787745 29633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83c9d5fc12064264a4afc721f5c87253" candidate_uuid: "3fae49eb096c4723af74c0400a310adb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a55e25464f6460f906b4a86a75baa9c"
I20260812 03:08:06.787916 29633 raft_consensus.cc:3037] T 83c9d5fc12064264a4afc721f5c87253 P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:06.788884 29783 raft_consensus.cc:2454] T 83c9d5fc12064264a4afc721f5c87253 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fae49eb096c4723af74c0400a310adb in term 1.
I20260812 03:08:06.789112 29633 raft_consensus.cc:2454] T 83c9d5fc12064264a4afc721f5c87253 P 7a55e25464f6460f906b4a86a75baa9c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fae49eb096c4723af74c0400a310adb in term 1.
I20260812 03:08:06.789261 29862 leader_election.cc:302] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fae49eb096c4723af74c0400a310adb, 5cfc8a046bde4bfc8649dd83b66bf0f2; no voters:
I20260812 03:08:06.789518 30021 raft_consensus.cc:2781] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:06.789654 30021 raft_consensus.cc:686] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb [term 1 LEADER]: Becoming Leader. State: Replica: 3fae49eb096c4723af74c0400a310adb, State: Running, Role: LEADER
I20260812 03:08:06.789810 30021 consensus_queue.cc:237] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:06.791905 29466 catalog_manager.cc:5495] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fae49eb096c4723af74c0400a310adb (127.26.250.131). New cstate: current_term: 1 leader_uuid: "3fae49eb096c4723af74c0400a310adb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } health_report { overall_health: HEALTHY } } }
I20260812 03:08:06.813519 29783 raft_consensus.cc:1261] T 83c9d5fc12064264a4afc721f5c87253 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 1 FOLLOWER]: Refusing update from remote peer 3fae49eb096c4723af74c0400a310adb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:06.813962 29633 raft_consensus.cc:1261] T 83c9d5fc12064264a4afc721f5c87253 P 7a55e25464f6460f906b4a86a75baa9c [term 1 FOLLOWER]: Refusing update from remote peer 3fae49eb096c4723af74c0400a310adb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:06.814412 30017 consensus_queue.cc:1035] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:06.814574 30021 consensus_queue.cc:1035] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 03:08:06.831810 30039 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 03:08:08.390563 29466 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:08.390758 29466 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:08.393451 29466 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69: Sending DeleteTablet for 3 replicas of tablet 83c9d5fc12064264a4afc721f5c87253
I20260812 03:08:08.394164 29611 tablet_service.cc:1514] Processing DeleteTablet for tablet 83c9d5fc12064264a4afc721f5c87253 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:08 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:46376
I20260812 03:08:08.394229 29760 tablet_service.cc:1514] Processing DeleteTablet for tablet 83c9d5fc12064264a4afc721f5c87253 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:08 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:38002
I20260812 03:08:08.394380 29912 tablet_service.cc:1514] Processing DeleteTablet for tablet 83c9d5fc12064264a4afc721f5c87253 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:08 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:52986
I20260812 03:08:08.394660 30089 tablet_replica.cc:330] T 83c9d5fc12064264a4afc721f5c87253 P 5cfc8a046bde4bfc8649dd83b66bf0f2: stopping tablet replica
I20260812 03:08:08.394831 30089 raft_consensus.cc:2229] T 83c9d5fc12064264a4afc721f5c87253 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:08.394845 30091 tablet_replica.cc:330] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb: stopping tablet replica
I20260812 03:08:08.394976 30089 raft_consensus.cc:2258] T 83c9d5fc12064264a4afc721f5c87253 P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:08.395004 30091 raft_consensus.cc:2229] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:08:08.395255 30091 raft_consensus.cc:2258] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:08.396065 30091 ts_tablet_manager.cc:1905] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:08.398669 30090 tablet_replica.cc:330] T 83c9d5fc12064264a4afc721f5c87253 P 7a55e25464f6460f906b4a86a75baa9c: stopping tablet replica
I20260812 03:08:08.398826 30090 raft_consensus.cc:2229] T 83c9d5fc12064264a4afc721f5c87253 P 7a55e25464f6460f906b4a86a75baa9c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:08.398866 30089 ts_tablet_manager.cc:1905] T 83c9d5fc12064264a4afc721f5c87253 P 5cfc8a046bde4bfc8649dd83b66bf0f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:08.398972 30090 raft_consensus.cc:2258] T 83c9d5fc12064264a4afc721f5c87253 P 7a55e25464f6460f906b4a86a75baa9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:08.399341 30091 ts_tablet_manager.cc:1918] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:08:08.399434 30091 log.cc:1196] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/83c9d5fc12064264a4afc721f5c87253
I20260812 03:08:08.399747 30091 ts_tablet_manager.cc:1939] T 83c9d5fc12064264a4afc721f5c87253 P 3fae49eb096c4723af74c0400a310adb: Deleting consensus metadata
I20260812 03:08:08.400650 29452 catalog_manager.cc:4841] TS 3fae49eb096c4723af74c0400a310adb (127.26.250.131:38449): tablet 83c9d5fc12064264a4afc721f5c87253 (table test-table [id=84b7d2d2cc4c4758850922efc951a82c]) successfully deleted
I20260812 03:08:08.402415 30089 ts_tablet_manager.cc:1918] T 83c9d5fc12064264a4afc721f5c87253 P 5cfc8a046bde4bfc8649dd83b66bf0f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:08:08.402513 30089 log.cc:1196] T 83c9d5fc12064264a4afc721f5c87253 P 5cfc8a046bde4bfc8649dd83b66bf0f2: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-1/wal/wals/83c9d5fc12064264a4afc721f5c87253
I20260812 03:08:08.402967 30089 ts_tablet_manager.cc:1939] T 83c9d5fc12064264a4afc721f5c87253 P 5cfc8a046bde4bfc8649dd83b66bf0f2: Deleting consensus metadata
I20260812 03:08:08.403064 30090 ts_tablet_manager.cc:1905] T 83c9d5fc12064264a4afc721f5c87253 P 7a55e25464f6460f906b4a86a75baa9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:08.403839 29452 catalog_manager.cc:4841] TS 5cfc8a046bde4bfc8649dd83b66bf0f2 (127.26.250.130:32889): tablet 83c9d5fc12064264a4afc721f5c87253 (table test-table [id=84b7d2d2cc4c4758850922efc951a82c]) successfully deleted
I20260812 03:08:08.405576 30090 ts_tablet_manager.cc:1918] T 83c9d5fc12064264a4afc721f5c87253 P 7a55e25464f6460f906b4a86a75baa9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:08:08.405715 30090 log.cc:1196] T 83c9d5fc12064264a4afc721f5c87253 P 7a55e25464f6460f906b4a86a75baa9c: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-0/wal/wals/83c9d5fc12064264a4afc721f5c87253
I20260812 03:08:08.405944 30090 ts_tablet_manager.cc:1939] T 83c9d5fc12064264a4afc721f5c87253 P 7a55e25464f6460f906b4a86a75baa9c: Deleting consensus metadata
I20260812 03:08:08.406781 29453 catalog_manager.cc:4841] TS 7a55e25464f6460f906b4a86a75baa9c (127.26.250.129:44973): tablet 83c9d5fc12064264a4afc721f5c87253 (table test-table [id=84b7d2d2cc4c4758850922efc951a82c]) successfully deleted
Aug 12 03:08:08 dist-test-slave-zpfg krb5kdc[29420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504088, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:08:08 dist-test-slave-zpfg krb5kdc[29420](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504088, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:08.430658 30100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:08.419604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50814 (local address 127.26.250.190:37709)
0812 03:08:08.419958 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:08.419971 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:08.419991 (+ 20us) server_negotiation.cc:407] Connection header received
0812 03:08:08.420172 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:08.420177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:08.420240 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:08.420376 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:08.421221 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:08.421854 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:08.422902 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:08.423216 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:08.423352 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:08.426631 (+ 3279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:08.426634 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:08.426636 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:08.426668 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:08.428443 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:08.428979 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:08.428982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:08.428984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:08.429075 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:08.429551 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:08.429554 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:08.429555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:08.429771 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:08.429880 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:08.430301 (+ 421us) server_negotiation.cc:299] Negotiation successful
0812 03:08:08.430494 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":118,"threads_started":1}
I20260812 03:08:08.433801 29466 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50814:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:08:08.434401 29466 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:08:08.440402 29611 tablet_service.cc:1467] Processing CreateTablet for tablet 601f8a16d1504afe83d5c4398ed8c05d (DEFAULT_TABLE table=test-table [id=5e00c8e42eb641cc91076dc0aef6c24d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:08.440407 29760 tablet_service.cc:1467] Processing CreateTablet for tablet 601f8a16d1504afe83d5c4398ed8c05d (DEFAULT_TABLE table=test-table [id=5e00c8e42eb641cc91076dc0aef6c24d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:08.440582 29912 tablet_service.cc:1467] Processing CreateTablet for tablet 601f8a16d1504afe83d5c4398ed8c05d (DEFAULT_TABLE table=test-table [id=5e00c8e42eb641cc91076dc0aef6c24d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:08.440754 29760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 601f8a16d1504afe83d5c4398ed8c05d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:08.440754 29611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 601f8a16d1504afe83d5c4398ed8c05d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:08.440894 29912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 601f8a16d1504afe83d5c4398ed8c05d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:08.444315 30103 tablet_bootstrap.cc:492] T 601f8a16d1504afe83d5c4398ed8c05d P 3fae49eb096c4723af74c0400a310adb: Bootstrap starting.
I20260812 03:08:08.444315 30101 tablet_bootstrap.cc:492] T 601f8a16d1504afe83d5c4398ed8c05d P 5cfc8a046bde4bfc8649dd83b66bf0f2: Bootstrap starting.
I20260812 03:08:08.445226 30102 tablet_bootstrap.cc:492] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c: Bootstrap starting.
I20260812 03:08:08.445355 30103 tablet_bootstrap.cc:654] T 601f8a16d1504afe83d5c4398ed8c05d P 3fae49eb096c4723af74c0400a310adb: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:08.445446 30101 tablet_bootstrap.cc:654] T 601f8a16d1504afe83d5c4398ed8c05d P 5cfc8a046bde4bfc8649dd83b66bf0f2: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:08.446831 30103 tablet_bootstrap.cc:492] T 601f8a16d1504afe83d5c4398ed8c05d P 3fae49eb096c4723af74c0400a310adb: No bootstrap required, opened a new log
I20260812 03:08:08.446929 30103 ts_tablet_manager.cc:1397] T 601f8a16d1504afe83d5c4398ed8c05d P 3fae49eb096c4723af74c0400a310adb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:08:08.447186 30101 tablet_bootstrap.cc:492] T 601f8a16d1504afe83d5c4398ed8c05d P 5cfc8a046bde4bfc8649dd83b66bf0f2: No bootstrap required, opened a new log
I20260812 03:08:08.447283 30101 ts_tablet_manager.cc:1397] T 601f8a16d1504afe83d5c4398ed8c05d P 5cfc8a046bde4bfc8649dd83b66bf0f2: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 03:08:08.447377 30103 raft_consensus.cc:348] T 601f8a16d1504afe83d5c4398ed8c05d P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:08.447507 30103 raft_consensus.cc:374] T 601f8a16d1504afe83d5c4398ed8c05d P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:08.447537 30103 raft_consensus.cc:729] T 601f8a16d1504afe83d5c4398ed8c05d P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fae49eb096c4723af74c0400a310adb, State: Initialized, Role: FOLLOWER
I20260812 03:08:08.447688 30103 consensus_queue.cc:260] T 601f8a16d1504afe83d5c4398ed8c05d P 3fae49eb096c4723af74c0400a310adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:08.447821 30101 raft_consensus.cc:348] T 601f8a16d1504afe83d5c4398ed8c05d P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:08.447947 30103 ts_tablet_manager.cc:1428] T 601f8a16d1504afe83d5c4398ed8c05d P 3fae49eb096c4723af74c0400a310adb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:08:08.447952 30101 raft_consensus.cc:374] T 601f8a16d1504afe83d5c4398ed8c05d P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:08.447988 30101 raft_consensus.cc:729] T 601f8a16d1504afe83d5c4398ed8c05d P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cfc8a046bde4bfc8649dd83b66bf0f2, State: Initialized, Role: FOLLOWER
I20260812 03:08:08.448109 30101 consensus_queue.cc:260] T 601f8a16d1504afe83d5c4398ed8c05d P 5cfc8a046bde4bfc8649dd83b66bf0f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:08.448592 30101 ts_tablet_manager.cc:1428] T 601f8a16d1504afe83d5c4398ed8c05d P 5cfc8a046bde4bfc8649dd83b66bf0f2: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260812 03:08:08.448767 30102 tablet_bootstrap.cc:654] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:08.450999 30102 tablet_bootstrap.cc:492] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c: No bootstrap required, opened a new log
I20260812 03:08:08.451109 30102 ts_tablet_manager.cc:1397] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260812 03:08:08.451534 30102 raft_consensus.cc:348] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:08.451660 30102 raft_consensus.cc:374] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:08.451706 30102 raft_consensus.cc:729] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a55e25464f6460f906b4a86a75baa9c, State: Initialized, Role: FOLLOWER
I20260812 03:08:08.451887 30102 consensus_queue.cc:260] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:08.452325 30102 ts_tablet_manager.cc:1428] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:08:08.459380 30108 raft_consensus.cc:482] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:08:08.459511 30108 raft_consensus.cc:504] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:08.460608 30108 leader_election.cc:288] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cfc8a046bde4bfc8649dd83b66bf0f2 (127.26.250.130:32889), 3fae49eb096c4723af74c0400a310adb (127.26.250.131:38449)
I20260812 03:08:08.465641 30112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:08.461382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:36977 (local address 127.26.250.131:38449)
0812 03:08:08.461771 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:08.461778 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:08.461793 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:08:08.461863 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:08.461868 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:08.461924 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:08.462015 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:08.462433 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:08.463008 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:08.464565 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:08.465025 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:08.465178 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:08.465238 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:08.465422 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 03:08:08.465510 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":113,"threads_started":1}
I20260812 03:08:08.465641 30111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:08.461257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:38449 (local address 127.26.250.129:36977)
0812 03:08:08.461646 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:08.461676 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:08.461765 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:08.462025 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:08.462031 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:08.462053 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:08.462254 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:08.462264 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:08.463275 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:08.463278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:08.464405 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:08.464436 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:08.465179 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:08.465183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:08.465288 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:08.465309 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 03:08:08.465401 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 03:08:08.465483 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":66,"thread_start_us":174,"threads_started":1}
I20260812 03:08:08.465756 30110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:08.461072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:35161 (local address 127.26.250.130:32889)
0812 03:08:08.461611 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:08.461622 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:08.461638 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:08:08.461702 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:08.461721 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:08.461781 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:08.461880 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:08.462279 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:08.463015 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:08.464591 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:08.465083 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:08.465249 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:08.465309 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:08.465495 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 03:08:08.465577 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":291,"threads_started":1}
I20260812 03:08:08.466125 30109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:08.460866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:32889 (local address 127.26.250.129:35161)
0812 03:08:08.461174 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:08.461195 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:08.461315 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:08.461910 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:08.461914 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:08.461933 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:08.462148 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:08.462155 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:08.463173 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:08.463178 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:08.464443 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:08.464453 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:08.465230 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:08.465235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:08.465348 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:08.465378 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 03:08:08.465476 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 03:08:08.465557 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":71,"thread_start_us":115,"threads_started":1}
I20260812 03:08:08.466475 29935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "601f8a16d1504afe83d5c4398ed8c05d" candidate_uuid: "7a55e25464f6460f906b4a86a75baa9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fae49eb096c4723af74c0400a310adb" is_pre_election: true
I20260812 03:08:08.466747 29935 raft_consensus.cc:2454] T 601f8a16d1504afe83d5c4398ed8c05d P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a55e25464f6460f906b4a86a75baa9c in term 0.
I20260812 03:08:08.467171 29562 leader_election.cc:302] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fae49eb096c4723af74c0400a310adb, 7a55e25464f6460f906b4a86a75baa9c; no voters:
I20260812 03:08:08.467515 30108 raft_consensus.cc:2781] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:08:08.467594 30108 raft_consensus.cc:482] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:08:08.467643 30108 raft_consensus.cc:3037] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:08.467825 29783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "601f8a16d1504afe83d5c4398ed8c05d" candidate_uuid: "7a55e25464f6460f906b4a86a75baa9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" is_pre_election: true
I20260812 03:08:08.467998 29783 raft_consensus.cc:2454] T 601f8a16d1504afe83d5c4398ed8c05d P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a55e25464f6460f906b4a86a75baa9c in term 0.
I20260812 03:08:08.469103 30108 raft_consensus.cc:504] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:08.469547 30108 leader_election.cc:288] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c [CANDIDATE]: Term 1 election: Requested vote from peers 5cfc8a046bde4bfc8649dd83b66bf0f2 (127.26.250.130:32889), 3fae49eb096c4723af74c0400a310adb (127.26.250.131:38449)
I20260812 03:08:08.470033 29935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "601f8a16d1504afe83d5c4398ed8c05d" candidate_uuid: "7a55e25464f6460f906b4a86a75baa9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fae49eb096c4723af74c0400a310adb"
I20260812 03:08:08.470067 29783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "601f8a16d1504afe83d5c4398ed8c05d" candidate_uuid: "7a55e25464f6460f906b4a86a75baa9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2"
I20260812 03:08:08.470148 29935 raft_consensus.cc:3037] T 601f8a16d1504afe83d5c4398ed8c05d P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:08.470230 29783 raft_consensus.cc:3037] T 601f8a16d1504afe83d5c4398ed8c05d P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:08.471308 29783 raft_consensus.cc:2454] T 601f8a16d1504afe83d5c4398ed8c05d P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a55e25464f6460f906b4a86a75baa9c in term 1.
I20260812 03:08:08.471326 29935 raft_consensus.cc:2454] T 601f8a16d1504afe83d5c4398ed8c05d P 3fae49eb096c4723af74c0400a310adb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a55e25464f6460f906b4a86a75baa9c in term 1.
I20260812 03:08:08.471648 29562 leader_election.cc:302] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cfc8a046bde4bfc8649dd83b66bf0f2, 7a55e25464f6460f906b4a86a75baa9c; no voters:
I20260812 03:08:08.471869 30108 raft_consensus.cc:2781] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:08.472126 30108 raft_consensus.cc:686] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c [term 1 LEADER]: Becoming Leader. State: Replica: 7a55e25464f6460f906b4a86a75baa9c, State: Running, Role: LEADER
I20260812 03:08:08.472441 30108 consensus_queue.cc:237] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:08.475145 29466 catalog_manager.cc:5495] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a55e25464f6460f906b4a86a75baa9c (127.26.250.129). New cstate: current_term: 1 leader_uuid: "7a55e25464f6460f906b4a86a75baa9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260812 03:08:08.501338 30116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:08.496475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43110 (local address 127.26.250.129:44973)
0812 03:08:08.496765 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:08.496775 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:08.496787 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:08:08.496870 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:08.496880 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:08.496938 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:08.497045 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:08:08.497345 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:08.498074 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:08.499366 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:08.499823 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:08.499922 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:08.500272 (+ 350us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:08:08.500832 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:08:08.500934 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:08.501069 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 03:08:08.501125 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":121,"threads_started":1}
I20260812 03:08:08.513999 30119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:08.510004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:38449 (local address 127.26.250.129:49133)
0812 03:08:08.510440 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:08.510461 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:08.510607 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:08.510895 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:08.510898 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:08.510906 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:08:08.511045 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:08.511051 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:08.511914 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:08.511919 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:08.512593 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:08.512602 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:08.513115 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:08.513119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:08.513224 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:08.513277 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:08:08.513623 (+ 346us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:08:08.513632 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 03:08:08.513723 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 03:08:08.513813 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":150,"threads_started":1}
I20260812 03:08:08.513978 30112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:08.510198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:49133 (local address 127.26.250.131:38449)
0812 03:08:08.510400 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:08.510406 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:08.510541 (+ 135us) server_negotiation.cc:407] Connection header received
0812 03:08:08.510724 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:08.510727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:08.510786 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:08.510880 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:08:08.511172 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:08.511731 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:08.512730 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:08.512986 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:08.513106 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:08.513441 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:08:08.513532 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:08:08.513616 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:08.513738 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 03:08:08.513801 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 03:08:08.520974 29783 raft_consensus.cc:1261] T 601f8a16d1504afe83d5c4398ed8c05d P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 1 FOLLOWER]: Refusing update from remote peer 7a55e25464f6460f906b4a86a75baa9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:08.521199 29935 raft_consensus.cc:1261] T 601f8a16d1504afe83d5c4398ed8c05d P 3fae49eb096c4723af74c0400a310adb [term 1 FOLLOWER]: Refusing update from remote peer 7a55e25464f6460f906b4a86a75baa9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:08.521610 30108 consensus_queue.cc:1035] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:08.522199 30113 consensus_queue.cc:1035] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:08.542578 30125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:08.538635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:44973 (local address 127.26.250.131:46217)
0812 03:08:08.538954 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:08.538999 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:08.539120 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:08.539564 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:08.539581 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:08.539591 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:08:08.539701 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:08.539705 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:08.540544 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:08.540546 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:08.541249 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:08.541264 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:08.541767 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:08.541770 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:08.541870 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:08.541906 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:08:08.542242 (+ 336us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:08:08.542248 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:08:08.542308 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 03:08:08.542449 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":140,"threads_started":1}
I20260812 03:08:08.542613 30116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:08.538689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:46217 (local address 127.26.250.129:44973)
0812 03:08:08.538843 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:08.538849 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:08.539091 (+ 242us) server_negotiation.cc:407] Connection header received
0812 03:08:08.539234 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:08.539239 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:08.539304 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:08.539385 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:08:08.539849 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:08.540432 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:08.541403 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:08.541646 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:08.541763 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:08.542032 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:08:08.542114 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:08:08.542197 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:08.542380 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 03:08:08.542449 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 03:08:08.570186 29466 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:08.570420 29466 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:08.572826 29466 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69: Sending DeleteTablet for 3 replicas of tablet 601f8a16d1504afe83d5c4398ed8c05d
I20260812 03:08:08.573525 29611 tablet_service.cc:1514] Processing DeleteTablet for tablet 601f8a16d1504afe83d5c4398ed8c05d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:08 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:46376
I20260812 03:08:08.573576 29760 tablet_service.cc:1514] Processing DeleteTablet for tablet 601f8a16d1504afe83d5c4398ed8c05d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:08 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:38002
I20260812 03:08:08.573607 29912 tablet_service.cc:1514] Processing DeleteTablet for tablet 601f8a16d1504afe83d5c4398ed8c05d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:08 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:52986
I20260812 03:08:08.573797 30089 tablet_replica.cc:330] T 601f8a16d1504afe83d5c4398ed8c05d P 5cfc8a046bde4bfc8649dd83b66bf0f2: stopping tablet replica
I20260812 03:08:08.573839 30090 tablet_replica.cc:330] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c: stopping tablet replica
I20260812 03:08:08.573954 30089 raft_consensus.cc:2229] T 601f8a16d1504afe83d5c4398ed8c05d P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:08.573995 30090 raft_consensus.cc:2229] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:08:08.574092 30089 raft_consensus.cc:2258] T 601f8a16d1504afe83d5c4398ed8c05d P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:08.574288 30091 tablet_replica.cc:330] T 601f8a16d1504afe83d5c4398ed8c05d P 3fae49eb096c4723af74c0400a310adb: stopping tablet replica
I20260812 03:08:08.574328 30090 raft_consensus.cc:2258] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:08.574460 30091 raft_consensus.cc:2229] T 601f8a16d1504afe83d5c4398ed8c05d P 3fae49eb096c4723af74c0400a310adb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:08.574600 30091 raft_consensus.cc:2258] T 601f8a16d1504afe83d5c4398ed8c05d P 3fae49eb096c4723af74c0400a310adb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:08.575155 30089 ts_tablet_manager.cc:1905] T 601f8a16d1504afe83d5c4398ed8c05d P 5cfc8a046bde4bfc8649dd83b66bf0f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:08.575312 30090 ts_tablet_manager.cc:1905] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:08.575610 30091 ts_tablet_manager.cc:1905] T 601f8a16d1504afe83d5c4398ed8c05d P 3fae49eb096c4723af74c0400a310adb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:08.576052 29466 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50814:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:08:08.576691 29466 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:08:08.577862 30089 ts_tablet_manager.cc:1918] T 601f8a16d1504afe83d5c4398ed8c05d P 5cfc8a046bde4bfc8649dd83b66bf0f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:08:08.577862 30090 ts_tablet_manager.cc:1918] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:08:08.577952 30090 log.cc:1196] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-0/wal/wals/601f8a16d1504afe83d5c4398ed8c05d
I20260812 03:08:08.577952 30089 log.cc:1196] T 601f8a16d1504afe83d5c4398ed8c05d P 5cfc8a046bde4bfc8649dd83b66bf0f2: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-1/wal/wals/601f8a16d1504afe83d5c4398ed8c05d
I20260812 03:08:08.578229 30090 ts_tablet_manager.cc:1939] T 601f8a16d1504afe83d5c4398ed8c05d P 7a55e25464f6460f906b4a86a75baa9c: Deleting consensus metadata
I20260812 03:08:08.578290 30089 ts_tablet_manager.cc:1939] T 601f8a16d1504afe83d5c4398ed8c05d P 5cfc8a046bde4bfc8649dd83b66bf0f2: Deleting consensus metadata
I20260812 03:08:08.578529 30091 ts_tablet_manager.cc:1918] T 601f8a16d1504afe83d5c4398ed8c05d P 3fae49eb096c4723af74c0400a310adb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:08:08.578608 30091 log.cc:1196] T 601f8a16d1504afe83d5c4398ed8c05d P 3fae49eb096c4723af74c0400a310adb: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/601f8a16d1504afe83d5c4398ed8c05d
I20260812 03:08:08.578869 30091 ts_tablet_manager.cc:1939] T 601f8a16d1504afe83d5c4398ed8c05d P 3fae49eb096c4723af74c0400a310adb: Deleting consensus metadata
I20260812 03:08:08.579229 29452 catalog_manager.cc:4841] TS 5cfc8a046bde4bfc8649dd83b66bf0f2 (127.26.250.130:32889): tablet 601f8a16d1504afe83d5c4398ed8c05d (table test-table [id=5e00c8e42eb641cc91076dc0aef6c24d]) successfully deleted
I20260812 03:08:08.579592 29452 catalog_manager.cc:4841] TS 3fae49eb096c4723af74c0400a310adb (127.26.250.131:38449): tablet 601f8a16d1504afe83d5c4398ed8c05d (table test-table [id=5e00c8e42eb641cc91076dc0aef6c24d]) successfully deleted
I20260812 03:08:08.579664 29453 catalog_manager.cc:4841] TS 7a55e25464f6460f906b4a86a75baa9c (127.26.250.129:44973): tablet 601f8a16d1504afe83d5c4398ed8c05d (table test-table [id=5e00c8e42eb641cc91076dc0aef6c24d]) successfully deleted
I20260812 03:08:08.582942 29760 tablet_service.cc:1467] Processing CreateTablet for tablet 9d3e2705edc54d39bf6582c3316d2e3f (DEFAULT_TABLE table=test-table [id=10ce13864d8840f8b81eed54c1647cc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:08.582930 29611 tablet_service.cc:1467] Processing CreateTablet for tablet 9d3e2705edc54d39bf6582c3316d2e3f (DEFAULT_TABLE table=test-table [id=10ce13864d8840f8b81eed54c1647cc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:08.583072 29912 tablet_service.cc:1467] Processing CreateTablet for tablet 9d3e2705edc54d39bf6582c3316d2e3f (DEFAULT_TABLE table=test-table [id=10ce13864d8840f8b81eed54c1647cc3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:08.583276 29760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d3e2705edc54d39bf6582c3316d2e3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:08.583280 29611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d3e2705edc54d39bf6582c3316d2e3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:08.583321 29912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d3e2705edc54d39bf6582c3316d2e3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:08.585840 30103 tablet_bootstrap.cc:492] T 9d3e2705edc54d39bf6582c3316d2e3f P 3fae49eb096c4723af74c0400a310adb: Bootstrap starting.
I20260812 03:08:08.586141 30102 tablet_bootstrap.cc:492] T 9d3e2705edc54d39bf6582c3316d2e3f P 7a55e25464f6460f906b4a86a75baa9c: Bootstrap starting.
I20260812 03:08:08.586841 30103 tablet_bootstrap.cc:654] T 9d3e2705edc54d39bf6582c3316d2e3f P 3fae49eb096c4723af74c0400a310adb: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:08.587312 30102 tablet_bootstrap.cc:654] T 9d3e2705edc54d39bf6582c3316d2e3f P 7a55e25464f6460f906b4a86a75baa9c: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:08.587602 30101 tablet_bootstrap.cc:492] T 9d3e2705edc54d39bf6582c3316d2e3f P 5cfc8a046bde4bfc8649dd83b66bf0f2: Bootstrap starting.
I20260812 03:08:08.588104 30103 tablet_bootstrap.cc:492] T 9d3e2705edc54d39bf6582c3316d2e3f P 3fae49eb096c4723af74c0400a310adb: No bootstrap required, opened a new log
I20260812 03:08:08.588191 30103 ts_tablet_manager.cc:1397] T 9d3e2705edc54d39bf6582c3316d2e3f P 3fae49eb096c4723af74c0400a310adb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:08:08.588640 30102 tablet_bootstrap.cc:492] T 9d3e2705edc54d39bf6582c3316d2e3f P 7a55e25464f6460f906b4a86a75baa9c: No bootstrap required, opened a new log
I20260812 03:08:08.588660 30101 tablet_bootstrap.cc:654] T 9d3e2705edc54d39bf6582c3316d2e3f P 5cfc8a046bde4bfc8649dd83b66bf0f2: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:08.588724 30102 ts_tablet_manager.cc:1397] T 9d3e2705edc54d39bf6582c3316d2e3f P 7a55e25464f6460f906b4a86a75baa9c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:08:08.588653 30103 raft_consensus.cc:348] T 9d3e2705edc54d39bf6582c3316d2e3f P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:08.588768 30103 raft_consensus.cc:374] T 9d3e2705edc54d39bf6582c3316d2e3f P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:08.588804 30103 raft_consensus.cc:729] T 9d3e2705edc54d39bf6582c3316d2e3f P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fae49eb096c4723af74c0400a310adb, State: Initialized, Role: FOLLOWER
I20260812 03:08:08.588943 30103 consensus_queue.cc:260] T 9d3e2705edc54d39bf6582c3316d2e3f P 3fae49eb096c4723af74c0400a310adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:08.589231 30103 ts_tablet_manager.cc:1428] T 9d3e2705edc54d39bf6582c3316d2e3f P 3fae49eb096c4723af74c0400a310adb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:08:08.589205 30102 raft_consensus.cc:348] T 9d3e2705edc54d39bf6582c3316d2e3f P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:08.589339 30102 raft_consensus.cc:374] T 9d3e2705edc54d39bf6582c3316d2e3f P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:08.589380 30102 raft_consensus.cc:729] T 9d3e2705edc54d39bf6582c3316d2e3f P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a55e25464f6460f906b4a86a75baa9c, State: Initialized, Role: FOLLOWER
I20260812 03:08:08.589514 30102 consensus_queue.cc:260] T 9d3e2705edc54d39bf6582c3316d2e3f P 7a55e25464f6460f906b4a86a75baa9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:08.589886 30102 ts_tablet_manager.cc:1428] T 9d3e2705edc54d39bf6582c3316d2e3f P 7a55e25464f6460f906b4a86a75baa9c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:08:08.590667 30101 tablet_bootstrap.cc:492] T 9d3e2705edc54d39bf6582c3316d2e3f P 5cfc8a046bde4bfc8649dd83b66bf0f2: No bootstrap required, opened a new log
I20260812 03:08:08.590754 30101 ts_tablet_manager.cc:1397] T 9d3e2705edc54d39bf6582c3316d2e3f P 5cfc8a046bde4bfc8649dd83b66bf0f2: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 03:08:08.591298 30101 raft_consensus.cc:348] T 9d3e2705edc54d39bf6582c3316d2e3f P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:08.591418 30101 raft_consensus.cc:374] T 9d3e2705edc54d39bf6582c3316d2e3f P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:08.591463 30101 raft_consensus.cc:729] T 9d3e2705edc54d39bf6582c3316d2e3f P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cfc8a046bde4bfc8649dd83b66bf0f2, State: Initialized, Role: FOLLOWER
I20260812 03:08:08.591630 30101 consensus_queue.cc:260] T 9d3e2705edc54d39bf6582c3316d2e3f P 5cfc8a046bde4bfc8649dd83b66bf0f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:08.591925 30101 ts_tablet_manager.cc:1428] T 9d3e2705edc54d39bf6582c3316d2e3f P 5cfc8a046bde4bfc8649dd83b66bf0f2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:08:08.653925 30126 raft_consensus.cc:482] T 9d3e2705edc54d39bf6582c3316d2e3f P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:08:08.654086 30126 raft_consensus.cc:504] T 9d3e2705edc54d39bf6582c3316d2e3f P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:08.654687 30126 leader_election.cc:288] T 9d3e2705edc54d39bf6582c3316d2e3f P 7a55e25464f6460f906b4a86a75baa9c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cfc8a046bde4bfc8649dd83b66bf0f2 (127.26.250.130:32889), 3fae49eb096c4723af74c0400a310adb (127.26.250.131:38449)
I20260812 03:08:08.655203 29935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3e2705edc54d39bf6582c3316d2e3f" candidate_uuid: "7a55e25464f6460f906b4a86a75baa9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fae49eb096c4723af74c0400a310adb" is_pre_election: true
I20260812 03:08:08.655203 29783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3e2705edc54d39bf6582c3316d2e3f" candidate_uuid: "7a55e25464f6460f906b4a86a75baa9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" is_pre_election: true
I20260812 03:08:08.655443 29783 raft_consensus.cc:2454] T 9d3e2705edc54d39bf6582c3316d2e3f P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a55e25464f6460f906b4a86a75baa9c in term 0.
I20260812 03:08:08.655812 29562 leader_election.cc:302] T 9d3e2705edc54d39bf6582c3316d2e3f P 7a55e25464f6460f906b4a86a75baa9c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cfc8a046bde4bfc8649dd83b66bf0f2, 7a55e25464f6460f906b4a86a75baa9c; no voters:
I20260812 03:08:08.655831 29935 raft_consensus.cc:2454] T 9d3e2705edc54d39bf6582c3316d2e3f P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a55e25464f6460f906b4a86a75baa9c in term 0.
I20260812 03:08:08.656011 30126 raft_consensus.cc:2781] T 9d3e2705edc54d39bf6582c3316d2e3f P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:08:08.656057 30126 raft_consensus.cc:482] T 9d3e2705edc54d39bf6582c3316d2e3f P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:08:08.656095 30126 raft_consensus.cc:3037] T 9d3e2705edc54d39bf6582c3316d2e3f P 7a55e25464f6460f906b4a86a75baa9c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:08.657487 30126 raft_consensus.cc:504] T 9d3e2705edc54d39bf6582c3316d2e3f P 7a55e25464f6460f906b4a86a75baa9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:08.657882 30126 leader_election.cc:288] T 9d3e2705edc54d39bf6582c3316d2e3f P 7a55e25464f6460f906b4a86a75baa9c [CANDIDATE]: Term 1 election: Requested vote from peers 5cfc8a046bde4bfc8649dd83b66bf0f2 (127.26.250.130:32889), 3fae49eb096c4723af74c0400a310adb (127.26.250.131:38449)
I20260812 03:08:08.658375 29783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3e2705edc54d39bf6582c3316d2e3f" candidate_uuid: "7a55e25464f6460f906b4a86a75baa9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2"
I20260812 03:08:08.658506 29783 raft_consensus.cc:3037] T 9d3e2705edc54d39bf6582c3316d2e3f P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:08.658613 29935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3e2705edc54d39bf6582c3316d2e3f" candidate_uuid: "7a55e25464f6460f906b4a86a75baa9c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fae49eb096c4723af74c0400a310adb"
I20260812 03:08:08.658767 29935 raft_consensus.cc:3037] T 9d3e2705edc54d39bf6582c3316d2e3f P 3fae49eb096c4723af74c0400a310adb [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:08.659667 29783 raft_consensus.cc:2454] T 9d3e2705edc54d39bf6582c3316d2e3f P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a55e25464f6460f906b4a86a75baa9c in term 1.
I20260812 03:08:08.659994 29935 raft_consensus.cc:2454] T 9d3e2705edc54d39bf6582c3316d2e3f P 3fae49eb096c4723af74c0400a310adb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a55e25464f6460f906b4a86a75baa9c in term 1.
I20260812 03:08:08.660012 29562 leader_election.cc:302] T 9d3e2705edc54d39bf6582c3316d2e3f P 7a55e25464f6460f906b4a86a75baa9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cfc8a046bde4bfc8649dd83b66bf0f2, 7a55e25464f6460f906b4a86a75baa9c; no voters:
I20260812 03:08:08.660219 30126 raft_consensus.cc:2781] T 9d3e2705edc54d39bf6582c3316d2e3f P 7a55e25464f6460f906b4a86a75baa9c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:08.660315 30126 raft_consensus.cc:686] T 9d3e2705edc54d39bf6582c3316d2e3f P 7a55e25464f6460f906b4a86a75baa9c [term 1 LEADER]: Becoming Leader. State: Replica: 7a55e25464f6460f906b4a86a75baa9c, State: Running, Role: LEADER
I20260812 03:08:08.660447 30126 consensus_queue.cc:237] T 9d3e2705edc54d39bf6582c3316d2e3f P 7a55e25464f6460f906b4a86a75baa9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } }
I20260812 03:08:08.662268 29468 catalog_manager.cc:5495] T 9d3e2705edc54d39bf6582c3316d2e3f P 7a55e25464f6460f906b4a86a75baa9c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a55e25464f6460f906b4a86a75baa9c (127.26.250.129). New cstate: current_term: 1 leader_uuid: "7a55e25464f6460f906b4a86a75baa9c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a55e25464f6460f906b4a86a75baa9c" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44973 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 } health_report { overall_health: UNKNOWN } } }
I20260812 03:08:08.689106 29782 raft_consensus.cc:1261] T 9d3e2705edc54d39bf6582c3316d2e3f P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 1 FOLLOWER]: Refusing update from remote peer 7a55e25464f6460f906b4a86a75baa9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:08.689657 30123 consensus_queue.cc:1035] T 9d3e2705edc54d39bf6582c3316d2e3f P 7a55e25464f6460f906b4a86a75baa9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cfc8a046bde4bfc8649dd83b66bf0f2" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 32889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:08.690220 29935 raft_consensus.cc:1261] T 9d3e2705edc54d39bf6582c3316d2e3f P 3fae49eb096c4723af74c0400a310adb [term 1 FOLLOWER]: Refusing update from remote peer 7a55e25464f6460f906b4a86a75baa9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:08.691223 30123 consensus_queue.cc:1035] T 9d3e2705edc54d39bf6582c3316d2e3f P 7a55e25464f6460f906b4a86a75baa9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fae49eb096c4723af74c0400a310adb" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 03:08:08.711629 30073 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 03:08:10.286690 29467 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:10.286870 29467 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:10.289973 29467 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 113aae1b1efb4a26876f54a569c6cf69: Sending DeleteTablet for 3 replicas of tablet 9d3e2705edc54d39bf6582c3316d2e3f
I20260812 03:08:10.290848 29760 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d3e2705edc54d39bf6582c3316d2e3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:10 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:38002
I20260812 03:08:10.290859 29611 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d3e2705edc54d39bf6582c3316d2e3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:10 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:46376
I20260812 03:08:10.290979 29912 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d3e2705edc54d39bf6582c3316d2e3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:10 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:52986
I20260812 03:08:10.291378 30157 tablet_replica.cc:330] T 9d3e2705edc54d39bf6582c3316d2e3f P 3fae49eb096c4723af74c0400a310adb: stopping tablet replica
I20260812 03:08:10.291378 30156 tablet_replica.cc:330] T 9d3e2705edc54d39bf6582c3316d2e3f P 7a55e25464f6460f906b4a86a75baa9c: stopping tablet replica
I20260812 03:08:10.291376 30155 tablet_replica.cc:330] T 9d3e2705edc54d39bf6582c3316d2e3f P 5cfc8a046bde4bfc8649dd83b66bf0f2: stopping tablet replica
I20260812 03:08:10.291595 30157 raft_consensus.cc:2229] T 9d3e2705edc54d39bf6582c3316d2e3f P 3fae49eb096c4723af74c0400a310adb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:10.291627 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 29543
I20260812 03:08:10.292615 30157 raft_consensus.cc:2258] T 9d3e2705edc54d39bf6582c3316d2e3f P 3fae49eb096c4723af74c0400a310adb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:10.294189 30157 ts_tablet_manager.cc:1905] T 9d3e2705edc54d39bf6582c3316d2e3f P 3fae49eb096c4723af74c0400a310adb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:10.291589 30155 raft_consensus.cc:2229] T 9d3e2705edc54d39bf6582c3316d2e3f P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:10.295840 30155 raft_consensus.cc:2258] T 9d3e2705edc54d39bf6582c3316d2e3f P 5cfc8a046bde4bfc8649dd83b66bf0f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:10.296772 30155 ts_tablet_manager.cc:1905] T 9d3e2705edc54d39bf6582c3316d2e3f P 5cfc8a046bde4bfc8649dd83b66bf0f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:10.297865 30157 ts_tablet_manager.cc:1918] T 9d3e2705edc54d39bf6582c3316d2e3f P 3fae49eb096c4723af74c0400a310adb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:08:10.297962 30157 log.cc:1196] T 9d3e2705edc54d39bf6582c3316d2e3f P 3fae49eb096c4723af74c0400a310adb: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/9d3e2705edc54d39bf6582c3316d2e3f
I20260812 03:08:10.298269 30157 ts_tablet_manager.cc:1939] T 9d3e2705edc54d39bf6582c3316d2e3f P 3fae49eb096c4723af74c0400a310adb: Deleting consensus metadata
I20260812 03:08:10.299346 29452 catalog_manager.cc:4841] TS 3fae49eb096c4723af74c0400a310adb (127.26.250.131:38449): tablet 9d3e2705edc54d39bf6582c3316d2e3f (table test-table [id=10ce13864d8840f8b81eed54c1647cc3]) successfully deleted
I20260812 03:08:10.299845 30155 ts_tablet_manager.cc:1918] T 9d3e2705edc54d39bf6582c3316d2e3f P 5cfc8a046bde4bfc8649dd83b66bf0f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 03:08:10.299917 30155 log.cc:1196] T 9d3e2705edc54d39bf6582c3316d2e3f P 5cfc8a046bde4bfc8649dd83b66bf0f2: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786504075277362-27626-0/minicluster-data/ts-1/wal/wals/9d3e2705edc54d39bf6582c3316d2e3f
I20260812 03:08:10.300187 30155 ts_tablet_manager.cc:1939] T 9d3e2705edc54d39bf6582c3316d2e3f P 5cfc8a046bde4bfc8649dd83b66bf0f2: Deleting consensus metadata
I20260812 03:08:10.301244 29452 catalog_manager.cc:4841] TS 5cfc8a046bde4bfc8649dd83b66bf0f2 (127.26.250.130:32889): tablet 9d3e2705edc54d39bf6582c3316d2e3f (table test-table [id=10ce13864d8840f8b81eed54c1647cc3]) successfully deleted
W20260812 03:08:10.304751 29453 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.250.129:44973 (error 108)
I20260812 03:08:10.305794 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 29691
I20260812 03:08:10.307616 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:10.305770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:44973 (local address 127.0.0.1:43120)
0812 03:08:10.307328 (+ 1558us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:10.307440 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.129:44973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1440,"thread_start_us":135,"threads_started":1}
W20260812 03:08:10.308116 29453 catalog_manager.cc:4584] TS 7a55e25464f6460f906b4a86a75baa9c (127.26.250.129:44973): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9d3e2705edc54d39bf6582c3316d2e3f: Network error: Client connection negotiation failed: client connection to 127.26.250.129:44973: connect: Connection refused (error 111)
I20260812 03:08:10.314846 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 29841
I20260812 03:08:10.325510 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 29433
2026-08-12T03:08:10Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6697 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:08:10 dist-test-slave-zpfg krb5kdc[30163](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:08:10 dist-test-slave-zpfg krb5kdc[30163](info): set up 2 sockets
Aug 12 03:08:10 dist-test-slave-zpfg krb5kdc[30163](info): commencing operation
krb5kdc: starting...
W20260812 03:08:11.428879 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:08:11 dist-test-slave-zpfg krb5kdc[30163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:08:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:08:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:08:11.570477 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:40585
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:40561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:40585
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:11.690763 30177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:11.691113 30177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:11.691207 30177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:11.695573 30177 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:08:11.695684 30177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:11.695719 30177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:11.695761 30177 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:08:11.695797 30177 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:08:11.701613 30177 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:40561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:40585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:40585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:11.702863 30177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:11.703982 30177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:11.713138 30183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:11.713146 30182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:11.713187 30185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:11.713519 30177 server_base.cc:1034] running on GCE node
I20260812 03:08:11.714246 30177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:11.715528 30177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:11.716799 30177 hybrid_clock.cc:648] HybridClock initialized: now 1786504091716765 us; error 49 us; skew 500 ppm
Aug 12 03:08:11 dist-test-slave-zpfg krb5kdc[30163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504091, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:11.727605 30177 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:08:11.728780 30177 webserver.cc:460] Webserver started at http://127.26.250.190:42417/ using document root <none> and password file <none>
I20260812 03:08:11.729398 30177 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:11.729462 30177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:11.729703 30177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:11.732029 30177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "781db84647d8465997af720c21cf5120"
format_stamp: "Formatted at 2026-08-12 03:08:11 on dist-test-slave-zpfg"
server_key: "2c2dbdf9280e761303b361cef559b87e"
server_key_iv: "4c4261ad064b9901af91b9636c3ee40e"
server_key_version: "encryptionkey@0"
I20260812 03:08:11.732688 30177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "781db84647d8465997af720c21cf5120"
format_stamp: "Formatted at 2026-08-12 03:08:11 on dist-test-slave-zpfg"
server_key: "2c2dbdf9280e761303b361cef559b87e"
server_key_iv: "4c4261ad064b9901af91b9636c3ee40e"
server_key_version: "encryptionkey@0"
I20260812 03:08:11.736982 30177 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:08:11.740000 30192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:11.741365 30177 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:08:11.741513 30177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "781db84647d8465997af720c21cf5120"
format_stamp: "Formatted at 2026-08-12 03:08:11 on dist-test-slave-zpfg"
server_key: "2c2dbdf9280e761303b361cef559b87e"
server_key_iv: "4c4261ad064b9901af91b9636c3ee40e"
server_key_version: "encryptionkey@0"
I20260812 03:08:11.741637 30177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:11.755071 30177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:11.758831 30177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:11.759155 30177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:11.767060 30177 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:40585
I20260812 03:08:11.767094 30253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:40585 every 8 connection(s)
I20260812 03:08:11.768148 30177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:08:11.768915 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 30177
I20260812 03:08:11.769043 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:08:11.769408 27626 external_mini_cluster.cc:1428] Setting key 060797d302245c3929994be4df739254
I20260812 03:08:11.771863 30254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:11.778414 30254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 781db84647d8465997af720c21cf5120: Bootstrap starting.
Aug 12 03:08:11 dist-test-slave-zpfg krb5kdc[30163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:11.781325 30254 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 781db84647d8465997af720c21cf5120: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:11.782120 30254 log.cc:824] T 00000000000000000000000000000000 P 781db84647d8465997af720c21cf5120: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:11.784363 30254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 781db84647d8465997af720c21cf5120: No bootstrap required, opened a new log
I20260812 03:08:11.785151 30257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:11.771057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57330 (local address 127.26.250.190:40585)
0812 03:08:11.771572 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:11.771594 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:11.771639 (+ 45us) server_negotiation.cc:407] Connection header received
0812 03:08:11.772485 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:11.772508 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:11.772895 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:11.773277 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:11.774200 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:11.775308 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:11.776233 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:11.776743 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:11.776925 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:11.779942 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:11.779948 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:11.779962 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:11.779996 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:11.781991 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:11.782607 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:11.782612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:11.782617 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:11.782738 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:11.783045 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:11.783051 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:11.783057 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:11.783491 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:11.783706 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:11.784017 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 03:08:11.784324 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":170,"threads_started":1}
I20260812 03:08:11.788228 30254 raft_consensus.cc:348] T 00000000000000000000000000000000 P 781db84647d8465997af720c21cf5120 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "781db84647d8465997af720c21cf5120" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 40585 } }
I20260812 03:08:11.788496 30254 raft_consensus.cc:374] T 00000000000000000000000000000000 P 781db84647d8465997af720c21cf5120 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:11.788579 30254 raft_consensus.cc:729] T 00000000000000000000000000000000 P 781db84647d8465997af720c21cf5120 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 781db84647d8465997af720c21cf5120, State: Initialized, Role: FOLLOWER
I20260812 03:08:11.789021 30254 consensus_queue.cc:260] T 00000000000000000000000000000000 P 781db84647d8465997af720c21cf5120 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "781db84647d8465997af720c21cf5120" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 40585 } }
I20260812 03:08:11.789175 30254 raft_consensus.cc:388] T 00000000000000000000000000000000 P 781db84647d8465997af720c21cf5120 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:08:11.789264 30254 raft_consensus.cc:482] T 00000000000000000000000000000000 P 781db84647d8465997af720c21cf5120 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:08:11.789388 30254 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 781db84647d8465997af720c21cf5120 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:11.790536 30254 raft_consensus.cc:504] T 00000000000000000000000000000000 P 781db84647d8465997af720c21cf5120 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "781db84647d8465997af720c21cf5120" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 40585 } }
I20260812 03:08:11.790907 30254 leader_election.cc:302] T 00000000000000000000000000000000 P 781db84647d8465997af720c21cf5120 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 781db84647d8465997af720c21cf5120; no voters:
I20260812 03:08:11.791272 30254 leader_election.cc:288] T 00000000000000000000000000000000 P 781db84647d8465997af720c21cf5120 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:08:11.791508 30259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 781db84647d8465997af720c21cf5120 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:11.791823 30259 raft_consensus.cc:686] T 00000000000000000000000000000000 P 781db84647d8465997af720c21cf5120 [term 1 LEADER]: Becoming Leader. State: Replica: 781db84647d8465997af720c21cf5120, State: Running, Role: LEADER
I20260812 03:08:11.792191 30259 consensus_queue.cc:237] T 00000000000000000000000000000000 P 781db84647d8465997af720c21cf5120 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "781db84647d8465997af720c21cf5120" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 40585 } }
I20260812 03:08:11.792799 30254 sys_catalog.cc:564] T 00000000000000000000000000000000 P 781db84647d8465997af720c21cf5120 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:08:11.794081 30261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 781db84647d8465997af720c21cf5120 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 781db84647d8465997af720c21cf5120. Latest consensus state: current_term: 1 leader_uuid: "781db84647d8465997af720c21cf5120" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "781db84647d8465997af720c21cf5120" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 40585 } } }
I20260812 03:08:11.794234 30261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 781db84647d8465997af720c21cf5120 [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:11.794587 30260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 781db84647d8465997af720c21cf5120 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "781db84647d8465997af720c21cf5120" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "781db84647d8465997af720c21cf5120" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 40585 } } }
I20260812 03:08:11.794684 30260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 781db84647d8465997af720c21cf5120 [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:11.794813 30268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:08:11.798028 30268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:08:11.803902 30268 catalog_manager.cc:1347] Generated new cluster ID: d6eb0859f3a8408e98823fd1cdb6fcf8
I20260812 03:08:11.803996 30268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:08:11.814973 30268 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:08:11.816035 30268 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:08:11.826526 30268 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 781db84647d8465997af720c21cf5120: Generated new TSK 0
I20260812 03:08:11.827334 30268 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.250.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
I20260812 03:08:11.866358 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:0
--local_ip_for_outbound_sockets=127.26.250.129
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:40585
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:40561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:11.997550 30280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:11.997918 30280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:11.997998 30280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:12.003265 30280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:12.003396 30280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:12.003504 30280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:08:12.009608 30280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:40561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:40585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:12.011065 30280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:12.012279 30280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:12.023733 30285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:12.023800 30286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:12.023950 30280 server_base.cc:1034] running on GCE node
W20260812 03:08:12.023756 30288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:12.024601 30280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:12.025409 30280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:12.026646 30280 hybrid_clock.cc:648] HybridClock initialized: now 1786504092026624 us; error 35 us; skew 500 ppm
Aug 12 03:08:12 dist-test-slave-zpfg krb5kdc[30163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504092, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:12.030202 30280 init.cc:377] Logged in from keytab as kudu/127.26.250.129@KRBTEST.COM (short username kudu)
I20260812 03:08:12.031333 30280 webserver.cc:460] Webserver started at http://127.26.250.129:42807/ using document root <none> and password file <none>
I20260812 03:08:12.031989 30280 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:12.032061 30280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:12.032347 30280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:12.034823 30280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/data/instance:
uuid: "3d7968ecc98a40fb98c5b32d44a0b7c0"
format_stamp: "Formatted at 2026-08-12 03:08:12 on dist-test-slave-zpfg"
server_key: "28acc32388fa163a82a9a75a67adcb7c"
server_key_iv: "0f9e9a6b25adb050eccb486bb7db771f"
server_key_version: "encryptionkey@0"
I20260812 03:08:12.035494 30280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance:
uuid: "3d7968ecc98a40fb98c5b32d44a0b7c0"
format_stamp: "Formatted at 2026-08-12 03:08:12 on dist-test-slave-zpfg"
server_key: "28acc32388fa163a82a9a75a67adcb7c"
server_key_iv: "0f9e9a6b25adb050eccb486bb7db771f"
server_key_version: "encryptionkey@0"
I20260812 03:08:12.039825 30280 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:08:12.042912 30295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:12.044427 30280 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 03:08:12.044597 30280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "3d7968ecc98a40fb98c5b32d44a0b7c0"
format_stamp: "Formatted at 2026-08-12 03:08:12 on dist-test-slave-zpfg"
server_key: "28acc32388fa163a82a9a75a67adcb7c"
server_key_iv: "0f9e9a6b25adb050eccb486bb7db771f"
server_key_version: "encryptionkey@0"
I20260812 03:08:12.044736 30280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:12.063640 30280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:12.067517 30280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:12.067754 30280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:12.068557 30280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:12.069497 30280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:12.069574 30280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:12.069646 30280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:12.069696 30280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:12.079870 30280 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:36021
I20260812 03:08:12.079905 30415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:36021 every 8 connection(s)
I20260812 03:08:12.080870 30280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:08:12.085512 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 30280
I20260812 03:08:12.085646 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance
I20260812 03:08:12.086033 27626 external_mini_cluster.cc:1428] Setting key 0286e909a2d03c10a8838d704d87e156
Aug 12 03:08:12 dist-test-slave-zpfg krb5kdc[30163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504092, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.250.130@KRBTEST.COM; defaulting to no policy
I20260812 03:08:12.097357 30257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:12.082944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:37433 (local address 127.26.250.190:40585)
0812 03:08:12.083108 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:12.083111 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:12.083962 (+ 851us) server_negotiation.cc:407] Connection header received
0812 03:08:12.085429 (+ 1467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:12.085433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:12.085499 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:12.085611 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:12.087411 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:12.088152 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:12.089482 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:12.089838 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:12.089967 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:12.092847 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:12.092851 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:12.092853 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:12.092881 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:12.094805 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:12.095443 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:12.095447 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:12.095449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:12.095539 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:12.095925 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:12.095930 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:12.095932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:12.096171 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:12.096419 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:12.097001 (+ 582us) server_negotiation.cc:299] Negotiation successful
0812 03:08:12.097164 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 03:08:12.098311 30418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:12.083302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:40585 (local address 127.26.250.129:37433)
0812 03:08:12.083791 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:12.083839 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:12.085172 (+ 1333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:12.085872 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:12.085881 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:12.086580 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:12.087174 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:12.087189 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:12.088329 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:12.088332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:12.089330 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:12.089341 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:12.090026 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:12.090030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:12.090219 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:12.090952 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:12.090971 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:12.092597 (+ 1626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:12.094997 (+ 2400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:12.095003 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:12.095021 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:12.095323 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:12.095638 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:12.095641 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:12.095643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:12.095820 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:12.096417 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:12.096425 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:12.096742 (+ 317us) client_negotiation.cc:769] Sending connection context
0812 03:08:12.096959 (+ 217us) client_negotiation.cc:240] Negotiation successful
0812 03:08:12.097345 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":127,"threads_started":1}
I20260812 03:08:12.099869 30416 heartbeater.cc:347] Connected to a master server at 127.26.250.190:40585
I20260812 03:08:12.100167 30416 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.130@KRBTEST.COM" created.
I20260812 03:08:12.100776 30416 heartbeater.cc:511] Master 127.26.250.190:40585 requested a full tablet report, sending...
I20260812 03:08:12.102690 30211 ts_manager.cc:194] Registered new tserver with Master: 3d7968ecc98a40fb98c5b32d44a0b7c0 (127.26.250.129:36021)
I20260812 03:08:12.103915 30211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.129@KRBTEST.COM'} at 127.26.250.129:37433
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
I20260812 03:08:12.120226 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:0
--local_ip_for_outbound_sockets=127.26.250.130
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:40585
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:40561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:12.246631 30422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:12.246989 30422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:12.247044 30422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:12.252152 30422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:12.252243 30422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:12.252316 30422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:08:12.258891 30422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:40561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:40585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:12.260172 30422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:12.261320 30422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:12.272830 30430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:12.272830 30428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:12.272878 30427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:12.273751 30422 server_base.cc:1034] running on GCE node
I20260812 03:08:12.274488 30422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:12.275277 30422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:12.276468 30422 hybrid_clock.cc:648] HybridClock initialized: now 1786504092276433 us; error 49 us; skew 500 ppm
Aug 12 03:08:12 dist-test-slave-zpfg krb5kdc[30163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504092, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:12.280071 30422 init.cc:377] Logged in from keytab as kudu/127.26.250.130@KRBTEST.COM (short username kudu)
I20260812 03:08:12.281416 30422 webserver.cc:460] Webserver started at http://127.26.250.130:37229/ using document root <none> and password file <none>
I20260812 03:08:12.282016 30422 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:12.282083 30422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:12.282282 30422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:12.284750 30422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/data/instance:
uuid: "cddfc8d0e3c94ef7840cced2865a7427"
format_stamp: "Formatted at 2026-08-12 03:08:12 on dist-test-slave-zpfg"
server_key: "43d6dea7ae68265d0fc513248f83c92e"
server_key_iv: "226425945ca0660294b50873fcb323c2"
server_key_version: "encryptionkey@0"
I20260812 03:08:12.285364 30422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance:
uuid: "cddfc8d0e3c94ef7840cced2865a7427"
format_stamp: "Formatted at 2026-08-12 03:08:12 on dist-test-slave-zpfg"
server_key: "43d6dea7ae68265d0fc513248f83c92e"
server_key_iv: "226425945ca0660294b50873fcb323c2"
server_key_version: "encryptionkey@0"
I20260812 03:08:12.290328 30422 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 03:08:12.293602 30438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:12.295482 30422 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:08:12.295614 30422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "cddfc8d0e3c94ef7840cced2865a7427"
format_stamp: "Formatted at 2026-08-12 03:08:12 on dist-test-slave-zpfg"
server_key: "43d6dea7ae68265d0fc513248f83c92e"
server_key_iv: "226425945ca0660294b50873fcb323c2"
server_key_version: "encryptionkey@0"
I20260812 03:08:12.295732 30422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:12.317281 30422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:12.321064 30422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:12.321250 30422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:12.321871 30422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:12.322857 30422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:12.322917 30422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:12.322962 30422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:12.322978 30422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:12.333976 30422 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:34329
I20260812 03:08:12.334064 30556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:34329 every 8 connection(s)
I20260812 03:08:12.335163 30422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:08:12.339228 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 30422
I20260812 03:08:12.339354 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance
I20260812 03:08:12.339710 27626 external_mini_cluster.cc:1428] Setting key 69fcf48d84420c7725ef390ea5a9e304
Aug 12 03:08:12 dist-test-slave-zpfg krb5kdc[30163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504092, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.250.131@KRBTEST.COM; defaulting to no policy
I20260812 03:08:12.352157 30257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:12.337394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:33545 (local address 127.26.250.190:40585)
0812 03:08:12.337551 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:12.337555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:12.338584 (+ 1029us) server_negotiation.cc:407] Connection header received
0812 03:08:12.340162 (+ 1578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:12.340167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:12.340230 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:12.340340 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:12.341863 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:12.342517 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:12.343816 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:12.344085 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:12.344201 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:12.347855 (+ 3654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:12.347859 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:12.347861 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:12.347891 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:12.349553 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:12.350282 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:12.350291 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:12.350292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:12.350443 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:12.350861 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:12.350864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:12.350866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:12.351157 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:12.351272 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:12.351837 (+ 565us) server_negotiation.cc:299] Negotiation successful
0812 03:08:12.351981 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:08:12.353289 30559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:12.337715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:40585 (local address 127.26.250.130:33545)
0812 03:08:12.338331 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:12.338469 (+ 138us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:12.339921 (+ 1452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:12.340560 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:12.340569 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:12.341148 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:12.341707 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:12.341722 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:12.342656 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:12.342661 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:12.343650 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:12.343665 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:12.344287 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:12.344293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:12.344546 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:12.345460 (+ 914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:12.345517 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:12.347615 (+ 2098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:12.349711 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:12.349718 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:12.349739 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:12.350156 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:12.350555 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:12.350558 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:12.350560 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:12.350733 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:12.351273 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:12.351281 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:12.351583 (+ 302us) client_negotiation.cc:769] Sending connection context
0812 03:08:12.351793 (+ 210us) client_negotiation.cc:240] Negotiation successful
0812 03:08:12.352193 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":146,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.131@KRBTEST.COM" created.
I20260812 03:08:12.354912 30557 heartbeater.cc:347] Connected to a master server at 127.26.250.190:40585
I20260812 03:08:12.355232 30557 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:12.356045 30557 heartbeater.cc:511] Master 127.26.250.190:40585 requested a full tablet report, sending...
I20260812 03:08:12.357524 30211 ts_manager.cc:194] Registered new tserver with Master: cddfc8d0e3c94ef7840cced2865a7427 (127.26.250.130:34329)
I20260812 03:08:12.358183 30211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.130@KRBTEST.COM'} at 127.26.250.130:33545
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
I20260812 03:08:12.375990 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:0
--local_ip_for_outbound_sockets=127.26.250.131
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:40585
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:40561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:12.500630 30562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:12.500974 30562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:12.501062 30562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:12.505347 30562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:12.505461 30562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:12.505563 30562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:08:12.512059 30562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:40561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:40585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:12.513298 30562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:12.514401 30562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:12.524916 30567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:12.524916 30568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:12.524977 30570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:12.525389 30562 server_base.cc:1034] running on GCE node
I20260812 03:08:12.525858 30562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:12.526624 30562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:12.527807 30562 hybrid_clock.cc:648] HybridClock initialized: now 1786504092527792 us; error 33 us; skew 500 ppm
Aug 12 03:08:12 dist-test-slave-zpfg krb5kdc[30163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504092, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:12.530913 30562 init.cc:377] Logged in from keytab as kudu/127.26.250.131@KRBTEST.COM (short username kudu)
I20260812 03:08:12.531962 30562 webserver.cc:460] Webserver started at http://127.26.250.131:37219/ using document root <none> and password file <none>
I20260812 03:08:12.532680 30562 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:12.532752 30562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:12.532979 30562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:12.535282 30562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/data/instance:
uuid: "dfd6635ba82f45f68815e894db5c1108"
format_stamp: "Formatted at 2026-08-12 03:08:12 on dist-test-slave-zpfg"
server_key: "5fab69ca4fd5f09756b993d1ee81e13e"
server_key_iv: "83e2a896d00b2a2dafb47941df9450fa"
server_key_version: "encryptionkey@0"
I20260812 03:08:12.535905 30562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance:
uuid: "dfd6635ba82f45f68815e894db5c1108"
format_stamp: "Formatted at 2026-08-12 03:08:12 on dist-test-slave-zpfg"
server_key: "5fab69ca4fd5f09756b993d1ee81e13e"
server_key_iv: "83e2a896d00b2a2dafb47941df9450fa"
server_key_version: "encryptionkey@0"
I20260812 03:08:12.540179 30562 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:08:12.543565 30578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:12.545302 30562 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:08:12.545462 30562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "dfd6635ba82f45f68815e894db5c1108"
format_stamp: "Formatted at 2026-08-12 03:08:12 on dist-test-slave-zpfg"
server_key: "5fab69ca4fd5f09756b993d1ee81e13e"
server_key_iv: "83e2a896d00b2a2dafb47941df9450fa"
server_key_version: "encryptionkey@0"
I20260812 03:08:12.545603 30562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:12.580188 30562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:12.584213 30562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:12.584456 30562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:12.585139 30562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:12.586226 30562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:12.586295 30562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:12.586380 30562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:12.586412 30562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:12.596743 30562 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:36451
I20260812 03:08:12.596762 30696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:36451 every 8 connection(s)
I20260812 03:08:12.597772 30562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:08:12.605552 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 30562
I20260812 03:08:12.605702 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance
I20260812 03:08:12.606073 27626 external_mini_cluster.cc:1428] Setting key 758143e065ffdabd7c93b9fbc4abcb14
Aug 12 03:08:12 dist-test-slave-zpfg krb5kdc[30163](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504092, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:12.614053 30257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:12.600090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:48297 (local address 127.26.250.190:40585)
0812 03:08:12.600254 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:12.600258 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:12.601199 (+ 941us) server_negotiation.cc:407] Connection header received
0812 03:08:12.602751 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:12.602755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:12.602829 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:12.602941 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:12.604246 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:12.604831 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:12.606566 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:12.606913 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:12.607031 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:12.610036 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:12.610039 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:12.610042 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:12.610071 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:12.611874 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:12.612430 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:12.612433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:12.612434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:12.612524 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:12.612907 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:12.612909 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:12.612910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:12.613117 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:12.613301 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:12.613776 (+ 475us) server_negotiation.cc:299] Negotiation successful
0812 03:08:12.613903 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 03:08:12.615051 30699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:12.600488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:40585 (local address 127.26.250.131:48297)
0812 03:08:12.601031 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:12.601079 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:12.602512 (+ 1433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:12.603140 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:12.603148 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:12.603552 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:12.604081 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:12.604094 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:12.604984 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:12.604988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:12.606415 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:12.606427 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:12.607095 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:12.607100 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:12.607332 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:12.608052 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:12.608070 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:12.609849 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:12.612002 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:12.612008 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:12.612023 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:12.612314 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:12.612625 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:12.612627 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:12.612629 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:12.612792 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:12.613236 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:12.613241 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:12.613536 (+ 295us) client_negotiation.cc:769] Sending connection context
0812 03:08:12.613710 (+ 174us) client_negotiation.cc:240] Negotiation successful
0812 03:08:12.614034 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":184,"threads_started":1}
I20260812 03:08:12.616598 30697 heartbeater.cc:347] Connected to a master server at 127.26.250.190:40585
I20260812 03:08:12.616905 30697 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:12.617439 30697 heartbeater.cc:511] Master 127.26.250.190:40585 requested a full tablet report, sending...
I20260812 03:08:12.618927 30211 ts_manager.cc:194] Registered new tserver with Master: dfd6635ba82f45f68815e894db5c1108 (127.26.250.131:36451)
I20260812 03:08:12.619678 30211 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.131@KRBTEST.COM'} at 127.26.250.131:48297
I20260812 03:08:12.620716 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 03:08:12.638432 30257 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:12.636691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57346 (local address 127.26.250.190:40585)
0812 03:08:12.636989 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:12.636994 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:12.637155 (+ 161us) server_negotiation.cc:407] Connection header received
0812 03:08:12.637273 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:12.637275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:12.637322 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:12.637400 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:12.638242 (+ 842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57346: BlockingRecv error: recv got EOF from 127.0.0.1:57346 (error 108)
Metrics: {"server-negotiator.queue_time_us":188}
I20260812 03:08:12.639505 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 30280
I20260812 03:08:12.649235 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 30422
I20260812 03:08:12.658836 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 30562
I20260812 03:08:12.666735 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 30177
2026-08-12T03:08:12Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2338 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:08:12 dist-test-slave-zpfg krb5kdc[30709](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:08:12 dist-test-slave-zpfg krb5kdc[30709](info): set up 2 sockets
Aug 12 03:08:12 dist-test-slave-zpfg krb5kdc[30709](info): commencing operation
krb5kdc: starting...
W20260812 03:08:13.790882 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:08:13 dist-test-slave-zpfg krb5kdc[30709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:08:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:08:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:08:13.934319 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:45445
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:42401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:45445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:14.058877 30723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:14.059253 30723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:14.059343 30723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:14.064285 30723 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:08:14.064380 30723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:14.064419 30723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:14.064457 30723 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:08:14.064491 30723 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:08:14.070282 30723 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:42401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:45445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:45445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:14.071573 30723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:14.072715 30723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:14.082087 30731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:14.082067 30723 server_base.cc:1034] running on GCE node
W20260812 03:08:14.082051 30729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:14.082300 30728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:14.083098 30723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:14.084362 30723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:14.085572 30723 hybrid_clock.cc:648] HybridClock initialized: now 1786504094085541 us; error 47 us; skew 500 ppm
Aug 12 03:08:14 dist-test-slave-zpfg krb5kdc[30709](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504094, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:14.089530 30723 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:08:14.090912 30723 webserver.cc:460] Webserver started at http://127.26.250.190:43087/ using document root <none> and password file <none>
I20260812 03:08:14.092031 30723 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:14.092180 30723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:14.092516 30723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:14.096297 30723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "1dbca0c32e28460483693884c76987bb"
format_stamp: "Formatted at 2026-08-12 03:08:14 on dist-test-slave-zpfg"
server_key: "c037f8e7269f66bdcdf181f5104cde00"
server_key_iv: "2ba7d5d1b80822a12a11dc2de1e3c90c"
server_key_version: "encryptionkey@0"
I20260812 03:08:14.097321 30723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "1dbca0c32e28460483693884c76987bb"
format_stamp: "Formatted at 2026-08-12 03:08:14 on dist-test-slave-zpfg"
server_key: "c037f8e7269f66bdcdf181f5104cde00"
server_key_iv: "2ba7d5d1b80822a12a11dc2de1e3c90c"
server_key_version: "encryptionkey@0"
I20260812 03:08:14.107287 30723 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.005s sys 0.000s
I20260812 03:08:14.111538 30738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:14.113529 30723 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260812 03:08:14.113729 30723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "1dbca0c32e28460483693884c76987bb"
format_stamp: "Formatted at 2026-08-12 03:08:14 on dist-test-slave-zpfg"
server_key: "c037f8e7269f66bdcdf181f5104cde00"
server_key_iv: "2ba7d5d1b80822a12a11dc2de1e3c90c"
server_key_version: "encryptionkey@0"
I20260812 03:08:14.113909 30723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:14.157328 30723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:14.161002 30723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:14.161182 30723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:14.170768 30723 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:45445
I20260812 03:08:14.171198 30793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:45445 every 8 connection(s)
I20260812 03:08:14.172334 30723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:08:14.173882 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 30723
I20260812 03:08:14.174026 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:08:14.174407 27626 external_mini_cluster.cc:1428] Setting key ea1dd2cd0cb54c97e7dbabdf3a66f42a
I20260812 03:08:14.176514 30794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:14.185875 30794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dbca0c32e28460483693884c76987bb: Bootstrap starting.
Aug 12 03:08:14 dist-test-slave-zpfg krb5kdc[30709](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:14.189309 30794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1dbca0c32e28460483693884c76987bb: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:14.190060 30794 log.cc:824] T 00000000000000000000000000000000 P 1dbca0c32e28460483693884c76987bb: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:14.192250 30794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dbca0c32e28460483693884c76987bb: No bootstrap required, opened a new log
I20260812 03:08:14.192781 30797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:14.176515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42392 (local address 127.26.250.190:45445)
0812 03:08:14.177079 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:14.177103 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:14.177154 (+ 51us) server_negotiation.cc:407] Connection header received
0812 03:08:14.178395 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:14.178425 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:14.178991 (+ 566us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:14.179436 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:14.181272 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:14.182619 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:14.183923 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:14.184442 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:14.184622 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:14.187722 (+ 3100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:14.187727 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:14.187741 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:14.187772 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:14.189514 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:14.189992 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:14.189998 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:14.190024 (+ 26us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:14.190180 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:14.190518 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:14.190522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:14.190524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:14.191058 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:14.191269 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:14.191579 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 03:08:14.191930 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":171,"threads_started":1}
I20260812 03:08:14.195763 30794 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1dbca0c32e28460483693884c76987bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dbca0c32e28460483693884c76987bb" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45445 } }
I20260812 03:08:14.196024 30794 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1dbca0c32e28460483693884c76987bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:14.196110 30794 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1dbca0c32e28460483693884c76987bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dbca0c32e28460483693884c76987bb, State: Initialized, Role: FOLLOWER
I20260812 03:08:14.196300 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:08:14.196645 30794 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1dbca0c32e28460483693884c76987bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dbca0c32e28460483693884c76987bb" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45445 } }
I20260812 03:08:14.196811 30794 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1dbca0c32e28460483693884c76987bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:08:14.196887 30794 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1dbca0c32e28460483693884c76987bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:08:14.197039 30794 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dbca0c32e28460483693884c76987bb [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:14.198609 30794 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1dbca0c32e28460483693884c76987bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dbca0c32e28460483693884c76987bb" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45445 } }
I20260812 03:08:14.199028 30794 leader_election.cc:302] T 00000000000000000000000000000000 P 1dbca0c32e28460483693884c76987bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1dbca0c32e28460483693884c76987bb; no voters:
I20260812 03:08:14.199394 30794 leader_election.cc:288] T 00000000000000000000000000000000 P 1dbca0c32e28460483693884c76987bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:08:14.199556 30800 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1dbca0c32e28460483693884c76987bb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:14.199888 30800 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1dbca0c32e28460483693884c76987bb [term 1 LEADER]: Becoming Leader. State: Replica: 1dbca0c32e28460483693884c76987bb, State: Running, Role: LEADER
I20260812 03:08:14.200286 30800 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dbca0c32e28460483693884c76987bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dbca0c32e28460483693884c76987bb" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45445 } }
I20260812 03:08:14.200840 30794 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1dbca0c32e28460483693884c76987bb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:08:14.201781 30800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dbca0c32e28460483693884c76987bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dbca0c32e28460483693884c76987bb. Latest consensus state: current_term: 1 leader_uuid: "1dbca0c32e28460483693884c76987bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dbca0c32e28460483693884c76987bb" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45445 } } }
I20260812 03:08:14.201918 30800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dbca0c32e28460483693884c76987bb [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:14.202894 30801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dbca0c32e28460483693884c76987bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1dbca0c32e28460483693884c76987bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dbca0c32e28460483693884c76987bb" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45445 } } }
I20260812 03:08:14.203008 30801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dbca0c32e28460483693884c76987bb [sys.catalog]: This master's current role is: LEADER
W20260812 03:08:14.207242 30815 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1dbca0c32e28460483693884c76987bb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:08:14.207340 30815 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:08:14.207408 30809 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:08:14.208345 30809 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:08:14.215034 30809 catalog_manager.cc:1347] Generated new cluster ID: 5fe6b56dc07e420bb08d440979d8ffbf
I20260812 03:08:14.215147 30809 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:08:14.221024 30797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:14.215654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42408 (local address 127.26.250.190:45445)
0812 03:08:14.215807 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:14.215813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:14.216136 (+ 323us) server_negotiation.cc:407] Connection header received
0812 03:08:14.216324 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:14.216327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:14.216375 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:14.216466 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:14.217026 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:14.217591 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:14.218517 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:14.218783 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:14.218904 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:14.219876 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:14.219880 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:14.219883 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:08:14.220371 (+ 488us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:14.220402 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:08:14.220418 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:14.220525 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:14.220761 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 03:08:14.220831 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:08:14.232565 30809 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:08:14.233755 30809 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:08:14.240213 30809 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1dbca0c32e28460483693884c76987bb: Generated new TSK 0
I20260812 03:08:14.241101 30809 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:08:14.251173 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 30723
2026-08-12T03:08:14Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1582 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:08:14 dist-test-slave-zpfg krb5kdc[30828](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:08:14 dist-test-slave-zpfg krb5kdc[30828](info): set up 2 sockets
Aug 12 03:08:14 dist-test-slave-zpfg krb5kdc[30828](info): commencing operation
krb5kdc: starting...
W20260812 03:08:15.366997 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:08:15 dist-test-slave-zpfg krb5kdc[30828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:08:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:08:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:08:15.509989 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:40391
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:44673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:40391
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:15.629575 30842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:15.629914 30842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:15.630012 30842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:15.634202 30842 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:08:15.634351 30842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:15.634394 30842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:15.634450 30842 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:08:15.634505 30842 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:08:15.640131 30842 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:44673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:40391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:40391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:15.641336 30842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:15.642505 30842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:15.651952 30848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:15.651926 30849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:15.652187 30842 server_base.cc:1034] running on GCE node
W20260812 03:08:15.652000 30852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:15.652915 30842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:15.654114 30842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:15.655320 30842 hybrid_clock.cc:648] HybridClock initialized: now 1786504095655308 us; error 66 us; skew 500 ppm
Aug 12 03:08:15 dist-test-slave-zpfg krb5kdc[30828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504095, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:15.658500 30842 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:08:15.659444 30842 webserver.cc:460] Webserver started at http://127.26.250.190:39575/ using document root <none> and password file <none>
I20260812 03:08:15.660037 30842 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:15.660135 30842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:15.660390 30842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:15.662582 30842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "95b2922c5bbb4617a609d02973fd9f65"
format_stamp: "Formatted at 2026-08-12 03:08:15 on dist-test-slave-zpfg"
server_key: "257c6cf70904fec4186c924e8e86a8ac"
server_key_iv: "24a8bfa4354d2eb62ee348adba3428e3"
server_key_version: "encryptionkey@0"
I20260812 03:08:15.663173 30842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "95b2922c5bbb4617a609d02973fd9f65"
format_stamp: "Formatted at 2026-08-12 03:08:15 on dist-test-slave-zpfg"
server_key: "257c6cf70904fec4186c924e8e86a8ac"
server_key_iv: "24a8bfa4354d2eb62ee348adba3428e3"
server_key_version: "encryptionkey@0"
I20260812 03:08:15.667207 30842 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:08:15.669786 30859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:15.671200 30842 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:08:15.671368 30842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "95b2922c5bbb4617a609d02973fd9f65"
format_stamp: "Formatted at 2026-08-12 03:08:15 on dist-test-slave-zpfg"
server_key: "257c6cf70904fec4186c924e8e86a8ac"
server_key_iv: "24a8bfa4354d2eb62ee348adba3428e3"
server_key_version: "encryptionkey@0"
I20260812 03:08:15.671494 30842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:15.694051 30842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:15.697779 30842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:15.698016 30842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:15.705600 30842 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:40391
I20260812 03:08:15.705958 30919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:40391 every 8 connection(s)
I20260812 03:08:15.706738 30842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:08:15.707937 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 30842
I20260812 03:08:15.708079 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:08:15.708367 27626 external_mini_cluster.cc:1428] Setting key 0f5646dd232ed4ee3246b864a4ac8286
I20260812 03:08:15.710264 30920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:15.717540 30920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95b2922c5bbb4617a609d02973fd9f65: Bootstrap starting.
Aug 12 03:08:15 dist-test-slave-zpfg krb5kdc[30828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:15.720573 30920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95b2922c5bbb4617a609d02973fd9f65: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:15.721395 30920 log.cc:824] T 00000000000000000000000000000000 P 95b2922c5bbb4617a609d02973fd9f65: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:15.724000 30920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95b2922c5bbb4617a609d02973fd9f65: No bootstrap required, opened a new log
I20260812 03:08:15.724466 30923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:15.709896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51170 (local address 127.26.250.190:40391)
0812 03:08:15.710372 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:15.710392 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:15.710444 (+ 52us) server_negotiation.cc:407] Connection header received
0812 03:08:15.711331 (+ 887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:15.711353 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:15.711784 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:15.712165 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:15.712876 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:15.714494 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:15.715439 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:15.716184 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:15.716360 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:15.719347 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:15.719352 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:15.719367 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:15.719398 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:15.721229 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:15.721751 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:15.721756 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:15.721761 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:15.721950 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:15.722270 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:15.722273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:15.722275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:15.722738 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:15.722900 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:15.723273 (+ 373us) server_negotiation.cc:299] Negotiation successful
0812 03:08:15.723572 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":127,"threads_started":1}
I20260812 03:08:15.727946 30920 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95b2922c5bbb4617a609d02973fd9f65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b2922c5bbb4617a609d02973fd9f65" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 40391 } }
I20260812 03:08:15.728190 30920 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95b2922c5bbb4617a609d02973fd9f65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:15.728273 30920 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95b2922c5bbb4617a609d02973fd9f65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95b2922c5bbb4617a609d02973fd9f65, State: Initialized, Role: FOLLOWER
I20260812 03:08:15.728781 30920 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95b2922c5bbb4617a609d02973fd9f65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b2922c5bbb4617a609d02973fd9f65" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 40391 } }
I20260812 03:08:15.728952 30920 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95b2922c5bbb4617a609d02973fd9f65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:08:15.729044 30920 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95b2922c5bbb4617a609d02973fd9f65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:08:15.729171 30920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95b2922c5bbb4617a609d02973fd9f65 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:15.730193 30920 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95b2922c5bbb4617a609d02973fd9f65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b2922c5bbb4617a609d02973fd9f65" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 40391 } }
I20260812 03:08:15.730582 30920 leader_election.cc:302] T 00000000000000000000000000000000 P 95b2922c5bbb4617a609d02973fd9f65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95b2922c5bbb4617a609d02973fd9f65; no voters:
I20260812 03:08:15.730969 30920 leader_election.cc:288] T 00000000000000000000000000000000 P 95b2922c5bbb4617a609d02973fd9f65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:08:15.731096 30925 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95b2922c5bbb4617a609d02973fd9f65 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:15.731377 30925 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95b2922c5bbb4617a609d02973fd9f65 [term 1 LEADER]: Becoming Leader. State: Replica: 95b2922c5bbb4617a609d02973fd9f65, State: Running, Role: LEADER
I20260812 03:08:15.731724 30925 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95b2922c5bbb4617a609d02973fd9f65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b2922c5bbb4617a609d02973fd9f65" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 40391 } }
I20260812 03:08:15.732379 30920 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95b2922c5bbb4617a609d02973fd9f65 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:08:15.733714 30928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95b2922c5bbb4617a609d02973fd9f65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95b2922c5bbb4617a609d02973fd9f65. Latest consensus state: current_term: 1 leader_uuid: "95b2922c5bbb4617a609d02973fd9f65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b2922c5bbb4617a609d02973fd9f65" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 40391 } } }
I20260812 03:08:15.733726 30926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95b2922c5bbb4617a609d02973fd9f65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95b2922c5bbb4617a609d02973fd9f65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b2922c5bbb4617a609d02973fd9f65" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 40391 } } }
I20260812 03:08:15.733918 30928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95b2922c5bbb4617a609d02973fd9f65 [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:15.733948 30926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95b2922c5bbb4617a609d02973fd9f65 [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:15.735042 30938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:08:15.737831 30938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:08:15.743705 30938 catalog_manager.cc:1347] Generated new cluster ID: 80c25043d3fd457b80c855e798f01d29
I20260812 03:08:15.743793 30938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:08:15.758590 30938 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:08:15.759613 30938 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:08:15.771030 30938 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95b2922c5bbb4617a609d02973fd9f65: Generated new TSK 0
I20260812 03:08:15.771926 30938 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.250.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
I20260812 03:08:15.816044 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:0
--local_ip_for_outbound_sockets=127.26.250.129
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:40391
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:44673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:15.937516 30948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:15.937796 30948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:15.937844 30948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:15.942090 30948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:15.942167 30948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:15.942229 30948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:08:15.947759 30948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:44673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:40391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:15.948920 30948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:15.950047 30948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:15.960538 30955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:15.960543 30954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:15.960541 30957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:15.960954 30948 server_base.cc:1034] running on GCE node
I20260812 03:08:15.961531 30948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:15.962216 30948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:15.963388 30948 hybrid_clock.cc:648] HybridClock initialized: now 1786504095963369 us; error 32 us; skew 500 ppm
Aug 12 03:08:15 dist-test-slave-zpfg krb5kdc[30828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504095, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:15.966629 30948 init.cc:377] Logged in from keytab as kudu/127.26.250.129@KRBTEST.COM (short username kudu)
I20260812 03:08:15.967720 30948 webserver.cc:460] Webserver started at http://127.26.250.129:39889/ using document root <none> and password file <none>
I20260812 03:08:15.968338 30948 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:15.968437 30948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:15.968684 30948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:15.970960 30948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/data/instance:
uuid: "9f9479ff249b4f3d85a5a7902c1ea063"
format_stamp: "Formatted at 2026-08-12 03:08:15 on dist-test-slave-zpfg"
server_key: "29df3c60292b0765df444337a2eeb051"
server_key_iv: "3434232e60a9dbae8b3028c43c1c0132"
server_key_version: "encryptionkey@0"
I20260812 03:08:15.971562 30948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance:
uuid: "9f9479ff249b4f3d85a5a7902c1ea063"
format_stamp: "Formatted at 2026-08-12 03:08:15 on dist-test-slave-zpfg"
server_key: "29df3c60292b0765df444337a2eeb051"
server_key_iv: "3434232e60a9dbae8b3028c43c1c0132"
server_key_version: "encryptionkey@0"
I20260812 03:08:15.975759 30948 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:08:15.978698 30964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:15.980067 30948 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:08:15.980206 30948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "9f9479ff249b4f3d85a5a7902c1ea063"
format_stamp: "Formatted at 2026-08-12 03:08:15 on dist-test-slave-zpfg"
server_key: "29df3c60292b0765df444337a2eeb051"
server_key_iv: "3434232e60a9dbae8b3028c43c1c0132"
server_key_version: "encryptionkey@0"
I20260812 03:08:15.980343 30948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:16.009593 30948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:16.013459 30948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:16.013700 30948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:16.014427 30948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:16.015333 30948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:16.015383 30948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:16.015455 30948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:16.015488 30948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:16.026013 30948 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:43173
I20260812 03:08:16.026034 31085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:43173 every 8 connection(s)
I20260812 03:08:16.027004 30948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:08:16.034509 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 30948
I20260812 03:08:16.034651 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance
I20260812 03:08:16.034956 27626 external_mini_cluster.cc:1428] Setting key 03f5164a03012d4ff56e691d88c49a7b
Aug 12 03:08:16 dist-test-slave-zpfg krb5kdc[30828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504095, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:16.042680 30923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:16.028995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:55693 (local address 127.26.250.190:40391)
0812 03:08:16.029170 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:16.029174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:16.030147 (+ 973us) server_negotiation.cc:407] Connection header received
0812 03:08:16.031672 (+ 1525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:16.031675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:16.031728 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:16.031813 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:16.033416 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:16.033968 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:16.035556 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:16.035851 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:16.035968 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:16.038806 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:16.038810 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:16.038813 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:16.038848 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:16.040430 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:16.041001 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:16.041005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:16.041006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:16.041084 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:16.041449 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:16.041452 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:16.041454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:16.041655 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:16.041768 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:16.042316 (+ 548us) server_negotiation.cc:299] Negotiation successful
0812 03:08:16.042493 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 03:08:16.043538 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:16.029394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:40391 (local address 127.26.250.129:55693)
0812 03:08:16.029961 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:16.030012 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:16.031374 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:16.032092 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:16.032109 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:16.032594 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:16.033272 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:16.033290 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:16.034163 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:16.034169 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:16.035419 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:16.035430 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:16.036034 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:16.036037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:16.036213 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:16.037002 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:16.037019 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:16.038618 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:16.040566 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:16.040572 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:16.040590 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:16.040878 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:16.041179 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:16.041183 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:16.041186 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:16.041341 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:16.041770 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:16.041780 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:16.042053 (+ 273us) client_negotiation.cc:769] Sending connection context
0812 03:08:16.042251 (+ 198us) client_negotiation.cc:240] Negotiation successful
0812 03:08:16.042629 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":133,"threads_started":1}
I20260812 03:08:16.044941 31086 heartbeater.cc:347] Connected to a master server at 127.26.250.190:40391
I20260812 03:08:16.045168 31086 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.26.250.130@KRBTEST.COM; defaulting to no policy
I20260812 03:08:16.045639 31086 heartbeater.cc:511] Master 127.26.250.190:40391 requested a full tablet report, sending...
I20260812 03:08:16.047350 30879 ts_manager.cc:194] Registered new tserver with Master: 9f9479ff249b4f3d85a5a7902c1ea063 (127.26.250.129:43173)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.130@KRBTEST.COM" created.
I20260812 03:08:16.048760 30879 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.129@KRBTEST.COM'} at 127.26.250.129:55693
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
I20260812 03:08:16.067523 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:0
--local_ip_for_outbound_sockets=127.26.250.130
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:40391
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:44673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:16.189667 31092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:16.189955 31092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:16.190001 31092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:16.193981 31092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:16.194056 31092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:16.194115 31092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:08:16.199631 31092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:44673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:40391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:16.200753 31092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:16.201881 31092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:16.212627 31098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:16.212656 31101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:16.212919 31092 server_base.cc:1034] running on GCE node
W20260812 03:08:16.213068 31099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:16.213657 31092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:16.214452 31092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:16.215682 31092 hybrid_clock.cc:648] HybridClock initialized: now 1786504096215646 us; error 53 us; skew 500 ppm
Aug 12 03:08:16 dist-test-slave-zpfg krb5kdc[30828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504096, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:16.218657 31092 init.cc:377] Logged in from keytab as kudu/127.26.250.130@KRBTEST.COM (short username kudu)
I20260812 03:08:16.219677 31092 webserver.cc:460] Webserver started at http://127.26.250.130:44405/ using document root <none> and password file <none>
I20260812 03:08:16.220288 31092 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:16.220391 31092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:16.220638 31092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:16.222934 31092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/data/instance:
uuid: "479c2f3f058b4a279faef0179d31c346"
format_stamp: "Formatted at 2026-08-12 03:08:16 on dist-test-slave-zpfg"
server_key: "38df29932e8c0b99b6d26028e2f38833"
server_key_iv: "d6674e6f5613fafa8f741b125219d7e0"
server_key_version: "encryptionkey@0"
I20260812 03:08:16.223536 31092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance:
uuid: "479c2f3f058b4a279faef0179d31c346"
format_stamp: "Formatted at 2026-08-12 03:08:16 on dist-test-slave-zpfg"
server_key: "38df29932e8c0b99b6d26028e2f38833"
server_key_iv: "d6674e6f5613fafa8f741b125219d7e0"
server_key_version: "encryptionkey@0"
I20260812 03:08:16.227707 31092 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:08:16.230602 31108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:16.232031 31092 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:08:16.232187 31092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "479c2f3f058b4a279faef0179d31c346"
format_stamp: "Formatted at 2026-08-12 03:08:16 on dist-test-slave-zpfg"
server_key: "38df29932e8c0b99b6d26028e2f38833"
server_key_iv: "d6674e6f5613fafa8f741b125219d7e0"
server_key_version: "encryptionkey@0"
I20260812 03:08:16.232321 31092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:16.253348 31092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:16.257010 31092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:16.257246 31092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:16.257925 31092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:16.258970 31092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:16.259054 31092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:16.259150 31092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:16.259202 31092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:16.269593 31092 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:36135
I20260812 03:08:16.269649 31228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:36135 every 8 connection(s)
I20260812 03:08:16.270761 31092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:08:16.276165 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 31092
I20260812 03:08:16.276373 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance
I20260812 03:08:16.276676 27626 external_mini_cluster.cc:1428] Setting key 12f503b904a621b39cf84a02c8d9a219
Aug 12 03:08:16 dist-test-slave-zpfg krb5kdc[30828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504096, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:16.290074 30923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:16.272951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:36125 (local address 127.26.250.190:40391)
0812 03:08:16.273116 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:16.273121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:16.274103 (+ 982us) server_negotiation.cc:407] Connection header received
0812 03:08:16.275789 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:16.275792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:16.275904 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:16.276009 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:16.277646 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:16.278488 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:16.280414 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:16.280829 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:16.280977 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:16.285027 (+ 4050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:16.285032 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:16.285035 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:16.285067 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:16.287458 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:16.288303 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:16.288307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:16.288309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:16.288428 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:16.288851 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:16.288855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:16.288857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:16.289064 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:16.289179 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:16.289738 (+ 559us) server_negotiation.cc:299] Negotiation successful
0812 03:08:16.289901 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
WARNING: no policy specified for kudu/127.26.250.131@KRBTEST.COM; defaulting to no policy
I20260812 03:08:16.291226 31231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:16.273440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:40391 (local address 127.26.250.130:36125)
0812 03:08:16.273933 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:16.273983 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:16.275574 (+ 1591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:16.276265 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:16.276276 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:16.276771 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:16.277445 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:16.277471 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:16.278654 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:16.278658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:16.280244 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:16.280262 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:16.281058 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:16.281065 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:16.281357 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:16.282149 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:16.282171 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:16.284761 (+ 2590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:16.287655 (+ 2894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:16.287663 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:16.287683 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:16.288156 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:16.288582 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:16.288584 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:16.288585 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:16.288745 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:16.289180 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:16.289188 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:16.289477 (+ 289us) client_negotiation.cc:769] Sending connection context
0812 03:08:16.289682 (+ 205us) client_negotiation.cc:240] Negotiation successful
0812 03:08:16.290074 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":128,"threads_started":1}
I20260812 03:08:16.292750 31229 heartbeater.cc:347] Connected to a master server at 127.26.250.190:40391
I20260812 03:08:16.293066 31229 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.131@KRBTEST.COM" created.
I20260812 03:08:16.293785 31229 heartbeater.cc:511] Master 127.26.250.190:40391 requested a full tablet report, sending...
I20260812 03:08:16.295241 30879 ts_manager.cc:194] Registered new tserver with Master: 479c2f3f058b4a279faef0179d31c346 (127.26.250.130:36135)
I20260812 03:08:16.296166 30879 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.130@KRBTEST.COM'} at 127.26.250.130:36125
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
I20260812 03:08:16.314850 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:0
--local_ip_for_outbound_sockets=127.26.250.131
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:40391
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:44673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:16.438138 31235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:16.438565 31235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:16.438618 31235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:16.442932 31235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:16.443010 31235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:16.443075 31235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:08:16.448678 31235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:44673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:40391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:16.449851 31235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:16.450969 31235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:16.461565 31243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:16.461448 31240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:16.461443 31241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:16.462050 31235 server_base.cc:1034] running on GCE node
I20260812 03:08:16.462579 31235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:16.463277 31235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:16.464480 31235 hybrid_clock.cc:648] HybridClock initialized: now 1786504096464441 us; error 52 us; skew 500 ppm
Aug 12 03:08:16 dist-test-slave-zpfg krb5kdc[30828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504096, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:16.467691 31235 init.cc:377] Logged in from keytab as kudu/127.26.250.131@KRBTEST.COM (short username kudu)
I20260812 03:08:16.468669 31235 webserver.cc:460] Webserver started at http://127.26.250.131:40035/ using document root <none> and password file <none>
I20260812 03:08:16.469292 31235 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:16.469446 31235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:16.469697 31235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:16.471944 31235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/data/instance:
uuid: "88364c52d56444659126fa00663dccee"
format_stamp: "Formatted at 2026-08-12 03:08:16 on dist-test-slave-zpfg"
server_key: "2189920c2a114b1396fb555a98b472a4"
server_key_iv: "7f08dce7b713b9cf4844ec9e65333ace"
server_key_version: "encryptionkey@0"
I20260812 03:08:16.472563 31235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance:
uuid: "88364c52d56444659126fa00663dccee"
format_stamp: "Formatted at 2026-08-12 03:08:16 on dist-test-slave-zpfg"
server_key: "2189920c2a114b1396fb555a98b472a4"
server_key_iv: "7f08dce7b713b9cf4844ec9e65333ace"
server_key_version: "encryptionkey@0"
I20260812 03:08:16.476999 31235 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:08:16.479875 31251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:16.481353 31235 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:08:16.481509 31235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "88364c52d56444659126fa00663dccee"
format_stamp: "Formatted at 2026-08-12 03:08:16 on dist-test-slave-zpfg"
server_key: "2189920c2a114b1396fb555a98b472a4"
server_key_iv: "7f08dce7b713b9cf4844ec9e65333ace"
server_key_version: "encryptionkey@0"
I20260812 03:08:16.481642 31235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:16.496078 31235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:16.499759 31235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:16.499986 31235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:16.500636 31235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:16.501518 31235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:16.501601 31235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:16.501693 31235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:16.501744 31235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:16.511781 31235 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:34783
I20260812 03:08:16.511806 31370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:34783 every 8 connection(s)
I20260812 03:08:16.512748 31235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:08:16.513244 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 31235
I20260812 03:08:16.513381 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance
I20260812 03:08:16.513713 27626 external_mini_cluster.cc:1428] Setting key 0ba3b826003b6139bcd17f70b29e588e
Aug 12 03:08:16 dist-test-slave-zpfg krb5kdc[30828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504096, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:16.527953 30923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:16.514930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:34991 (local address 127.26.250.190:40391)
0812 03:08:16.515066 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:16.515071 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:16.516142 (+ 1071us) server_negotiation.cc:407] Connection header received
0812 03:08:16.517684 (+ 1542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:16.517688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:16.517745 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:16.517826 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:16.519160 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:16.519715 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:16.520897 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:16.521131 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:16.521255 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:16.524066 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:16.524070 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:16.524072 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:16.524100 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:16.525453 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:16.526044 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:16.526048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:16.526050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:16.526140 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:16.526618 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:16.526621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:16.526623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:16.526826 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:16.526943 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:16.527643 (+ 700us) server_negotiation.cc:299] Negotiation successful
0812 03:08:16.527782 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:08:16.529016 31373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:16.515412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:40391 (local address 127.26.250.131:34991)
0812 03:08:16.515969 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:16.516016 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:16.517438 (+ 1422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:16.518009 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:16.518016 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:16.518482 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:16.519008 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:16.519022 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:16.519860 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:16.519863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:16.520759 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:16.520769 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:16.521295 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:16.521299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:16.521477 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:16.522143 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:16.522160 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:16.523863 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:16.525586 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:16.525593 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:16.525610 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:16.525929 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:16.526240 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:16.526245 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:16.526247 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:16.526494 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:16.526942 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:16.526949 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:16.527344 (+ 395us) client_negotiation.cc:769] Sending connection context
0812 03:08:16.527589 (+ 245us) client_negotiation.cc:240] Negotiation successful
0812 03:08:16.527966 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":180,"threads_started":1}
I20260812 03:08:16.530195 31371 heartbeater.cc:347] Connected to a master server at 127.26.250.190:40391
I20260812 03:08:16.530512 31371 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:16.531075 31371 heartbeater.cc:511] Master 127.26.250.190:40391 requested a full tablet report, sending...
I20260812 03:08:16.532186 30879 ts_manager.cc:194] Registered new tserver with Master: 88364c52d56444659126fa00663dccee (127.26.250.131:34783)
I20260812 03:08:16.532892 30879 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.131@KRBTEST.COM'} at 127.26.250.131:34991
I20260812 03:08:16.541066 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 03:08:16 dist-test-slave-zpfg krb5kdc[30828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504096, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 03:08:16 dist-test-slave-zpfg krb5kdc[30828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504096, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:16.570752 30923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:16.561526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51178 (local address 127.26.250.190:40391)
0812 03:08:16.561664 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:16.561668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:16.561773 (+ 105us) server_negotiation.cc:407] Connection header received
0812 03:08:16.561916 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:16.561919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:16.561967 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:16.562055 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:16.563005 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:16.563543 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:16.564484 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:16.564702 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:16.564786 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:16.567511 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:16.567514 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:16.567516 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:16.567546 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:16.569101 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:16.569525 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:16.569527 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:16.569529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:16.569610 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:16.569854 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:16.569857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:16.569858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:16.570090 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:16.570196 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:16.570506 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 03:08:16.570621 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260812 03:08:16.571291 30879 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:51178
I20260812 03:08:16.572453 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 30948
I20260812 03:08:16.581112 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 31092
I20260812 03:08:16.591897 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 31235
I20260812 03:08:16.599663 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 30842
2026-08-12T03:08:16Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2350 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:08:16 dist-test-slave-zpfg krb5kdc[31384](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:08:16 dist-test-slave-zpfg krb5kdc[31384](info): set up 2 sockets
Aug 12 03:08:16 dist-test-slave-zpfg krb5kdc[31384](info): commencing operation
krb5kdc: starting...
W20260812 03:08:17.707767 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:08:17 dist-test-slave-zpfg krb5kdc[31384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:08:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:08:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:08:17.843900 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:39453
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:41299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:39453
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:17.974233 31400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:17.974581 31400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:17.974627 31400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:17.980176 31400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:08:17.980261 31400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:17.980314 31400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:17.980348 31400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:08:17.980382 31400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:08:17.986819 31400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:41299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:39453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:39453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:17.988145 31400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:17.989324 31400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:17.999404 31406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:17.999508 31400 server_base.cc:1034] running on GCE node
W20260812 03:08:17.999495 31405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:17.999428 31408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:18.000432 31400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:18.001832 31400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:18.003062 31400 hybrid_clock.cc:648] HybridClock initialized: now 1786504098003030 us; error 48 us; skew 500 ppm
Aug 12 03:08:18 dist-test-slave-zpfg krb5kdc[31384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504098, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:18.014505 31400 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:08:18.015702 31400 webserver.cc:460] Webserver started at http://127.26.250.190:44525/ using document root <none> and password file <none>
I20260812 03:08:18.016434 31400 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:18.016556 31400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:18.016832 31400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:18.019377 31400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "7debf657590745c7835272301243734e"
format_stamp: "Formatted at 2026-08-12 03:08:18 on dist-test-slave-zpfg"
server_key: "969df5b78a682bdbe1c31702951ed036"
server_key_iv: "3f1d9b4032a977ee4df55ddfda6daff5"
server_key_version: "encryptionkey@0"
I20260812 03:08:18.020053 31400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "7debf657590745c7835272301243734e"
format_stamp: "Formatted at 2026-08-12 03:08:18 on dist-test-slave-zpfg"
server_key: "969df5b78a682bdbe1c31702951ed036"
server_key_iv: "3f1d9b4032a977ee4df55ddfda6daff5"
server_key_version: "encryptionkey@0"
I20260812 03:08:18.024763 31400 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:08:18.027810 31415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:18.029345 31400 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:08:18.029532 31400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "7debf657590745c7835272301243734e"
format_stamp: "Formatted at 2026-08-12 03:08:18 on dist-test-slave-zpfg"
server_key: "969df5b78a682bdbe1c31702951ed036"
server_key_iv: "3f1d9b4032a977ee4df55ddfda6daff5"
server_key_version: "encryptionkey@0"
I20260812 03:08:18.029675 31400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:18.040369 31400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:18.044116 31400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:18.044361 31400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:18.052680 31400 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:39453
I20260812 03:08:18.052691 31468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:39453 every 8 connection(s)
I20260812 03:08:18.053766 31400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:08:18.056918 31469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:18.062731 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 31400
I20260812 03:08:18.062755 31469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7debf657590745c7835272301243734e: Bootstrap starting.
I20260812 03:08:18.062935 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:08:18.063278 27626 external_mini_cluster.cc:1428] Setting key bcb7df9da04201f1cbe93d28bf34fa1c
I20260812 03:08:18.065412 31469 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7debf657590745c7835272301243734e: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:18.066198 31469 log.cc:824] T 00000000000000000000000000000000 P 7debf657590745c7835272301243734e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:18.068219 31469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7debf657590745c7835272301243734e: No bootstrap required, opened a new log
I20260812 03:08:18.072506 31469 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7debf657590745c7835272301243734e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7debf657590745c7835272301243734e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39453 } }
I20260812 03:08:18.072818 31469 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7debf657590745c7835272301243734e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:18.072906 31469 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7debf657590745c7835272301243734e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7debf657590745c7835272301243734e, State: Initialized, Role: FOLLOWER
Aug 12 03:08:18 dist-test-slave-zpfg krb5kdc[31384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:18.073431 31469 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7debf657590745c7835272301243734e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7debf657590745c7835272301243734e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39453 } }
I20260812 03:08:18.073621 31469 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7debf657590745c7835272301243734e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:08:18.073698 31469 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7debf657590745c7835272301243734e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:08:18.073822 31469 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7debf657590745c7835272301243734e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:18.074908 31469 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7debf657590745c7835272301243734e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7debf657590745c7835272301243734e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39453 } }
I20260812 03:08:18.075218 31469 leader_election.cc:302] T 00000000000000000000000000000000 P 7debf657590745c7835272301243734e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7debf657590745c7835272301243734e; no voters:
I20260812 03:08:18.075555 31469 leader_election.cc:288] T 00000000000000000000000000000000 P 7debf657590745c7835272301243734e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:08:18.075713 31474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7debf657590745c7835272301243734e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:18.076001 31474 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7debf657590745c7835272301243734e [term 1 LEADER]: Becoming Leader. State: Replica: 7debf657590745c7835272301243734e, State: Running, Role: LEADER
I20260812 03:08:18.076372 31474 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7debf657590745c7835272301243734e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7debf657590745c7835272301243734e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39453 } }
I20260812 03:08:18.077005 31469 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7debf657590745c7835272301243734e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:08:18.077773 31474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7debf657590745c7835272301243734e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7debf657590745c7835272301243734e. Latest consensus state: current_term: 1 leader_uuid: "7debf657590745c7835272301243734e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7debf657590745c7835272301243734e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39453 } } }
I20260812 03:08:18.077924 31474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7debf657590745c7835272301243734e [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:18.078460 31476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7debf657590745c7835272301243734e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7debf657590745c7835272301243734e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7debf657590745c7835272301243734e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39453 } } }
I20260812 03:08:18.078575 31476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7debf657590745c7835272301243734e [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:18.081153 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:18.064772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44410 (local address 127.26.250.190:39453)
0812 03:08:18.065238 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:18.065257 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:18.065296 (+ 39us) server_negotiation.cc:407] Connection header received
0812 03:08:18.066059 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:18.066080 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:18.066534 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:18.066915 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:18.067864 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:18.069013 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:18.069970 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:18.070760 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:18.071448 (+ 688us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:18.074188 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:18.074194 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:18.074209 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:18.074238 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:18.076612 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:18.077205 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:18.077211 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:18.077218 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:18.077397 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:18.078461 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:18.078465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:18.078467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:18.079060 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:18.079939 (+ 879us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:18.080136 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 03:08:18.080413 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":156,"threads_started":1}
W20260812 03:08:18.083498 31490 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7debf657590745c7835272301243734e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:08:18.083586 31490 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:08:18.083684 31484 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:08:18.084461 31484 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:08:18.090092 31484 catalog_manager.cc:1347] Generated new cluster ID: d41c30a800b1469e9f307a78a11d1c94
I20260812 03:08:18.090173 31484 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:08:18.100903 31484 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:08:18.102376 31484 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:08:18.107630 31484 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7debf657590745c7835272301243734e: Generated new TSK 0
I20260812 03:08:18.108518 31484 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.250.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
I20260812 03:08:18.142457 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:0
--local_ip_for_outbound_sockets=127.26.250.129
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39453
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:41299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:18.263850 31496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:18.264179 31496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:18.264227 31496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:18.268584 31496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:18.268671 31496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:18.268734 31496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:08:18.274549 31496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:41299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:18.275792 31496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:18.277025 31496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:18.288033 31501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:18.287988 31504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:18.287989 31502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:18.288339 31496 server_base.cc:1034] running on GCE node
I20260812 03:08:18.288827 31496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:18.289561 31496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:18.290758 31496 hybrid_clock.cc:648] HybridClock initialized: now 1786504098290742 us; error 28 us; skew 500 ppm
Aug 12 03:08:18 dist-test-slave-zpfg krb5kdc[31384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504098, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:18.293660 31496 init.cc:377] Logged in from keytab as kudu/127.26.250.129@KRBTEST.COM (short username kudu)
I20260812 03:08:18.294706 31496 webserver.cc:460] Webserver started at http://127.26.250.129:38673/ using document root <none> and password file <none>
I20260812 03:08:18.295332 31496 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:18.295433 31496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:18.295675 31496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:18.297947 31496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-0/data/instance:
uuid: "4893a8ea566e45eb8315ade7291a5650"
format_stamp: "Formatted at 2026-08-12 03:08:18 on dist-test-slave-zpfg"
server_key: "795bbd4a24b2b71dd758a7a59be6d9c2"
server_key_iv: "cc7b3ea0926a3da706784dbe90c7d7b1"
server_key_version: "encryptionkey@0"
I20260812 03:08:18.298664 31496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance:
uuid: "4893a8ea566e45eb8315ade7291a5650"
format_stamp: "Formatted at 2026-08-12 03:08:18 on dist-test-slave-zpfg"
server_key: "795bbd4a24b2b71dd758a7a59be6d9c2"
server_key_iv: "cc7b3ea0926a3da706784dbe90c7d7b1"
server_key_version: "encryptionkey@0"
I20260812 03:08:18.303149 31496 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:08:18.306233 31511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:18.307824 31496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:08:18.308002 31496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "4893a8ea566e45eb8315ade7291a5650"
format_stamp: "Formatted at 2026-08-12 03:08:18 on dist-test-slave-zpfg"
server_key: "795bbd4a24b2b71dd758a7a59be6d9c2"
server_key_iv: "cc7b3ea0926a3da706784dbe90c7d7b1"
server_key_version: "encryptionkey@0"
I20260812 03:08:18.308144 31496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:18.320693 31496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:18.324374 31496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:18.324607 31496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:18.325273 31496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:18.326162 31496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:18.326237 31496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:18.326308 31496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:18.326388 31496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260812 03:08:18.336851 31496 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:37751
I20260812 03:08:18.336862 31632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:37751 every 8 connection(s)
I20260812 03:08:18.337849 31496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:08:18.340685 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 31496
I20260812 03:08:18.340829 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance
I20260812 03:08:18.341190 27626 external_mini_cluster.cc:1428] Setting key 537197600e989d37fd728d8fb1ccf3e8
Aug 12 03:08:18 dist-test-slave-zpfg krb5kdc[31384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504098, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.250.130@KRBTEST.COM; defaulting to no policy
I20260812 03:08:18.354780 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:18.339781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:47231 (local address 127.26.250.190:39453)
0812 03:08:18.339951 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:18.339954 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:18.341131 (+ 1177us) server_negotiation.cc:407] Connection header received
0812 03:08:18.342979 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:18.342982 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:18.343027 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:18.343095 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:18.344824 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:18.345404 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:18.347108 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:18.347534 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:18.347665 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:18.350694 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:18.350701 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:18.350704 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:18.350737 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:18.352492 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:18.353089 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:18.353096 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:18.353101 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:18.353202 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:18.353585 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:18.353588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:18.353589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:18.353788 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:18.353876 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:18.354529 (+ 653us) server_negotiation.cc:299] Negotiation successful
0812 03:08:18.354649 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 03:08:18.355785 31635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:18.340255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39453 (local address 127.26.250.129:47231)
0812 03:08:18.340943 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:18.341012 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:18.342721 (+ 1709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:18.343294 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:18.343304 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:18.343895 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:18.344645 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:18.344668 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:18.345550 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:18.345556 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:18.346971 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:18.346981 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:18.347704 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:18.347706 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:18.347868 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:18.348581 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:18.348599 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:18.350484 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:18.352647 (+ 2163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:18.352653 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:18.352669 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:18.352959 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:18.353307 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:18.353310 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:18.353311 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:18.353479 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:18.353898 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:18.353904 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:18.354195 (+ 291us) client_negotiation.cc:769] Sending connection context
0812 03:08:18.354457 (+ 262us) client_negotiation.cc:240] Negotiation successful
0812 03:08:18.354756 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":138,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.130@KRBTEST.COM" created.
I20260812 03:08:18.357475 31633 heartbeater.cc:347] Connected to a master server at 127.26.250.190:39453
I20260812 03:08:18.357771 31633 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:18.358410 31633 heartbeater.cc:511] Master 127.26.250.190:39453 requested a full tablet report, sending...
I20260812 03:08:18.360267 31432 ts_manager.cc:194] Registered new tserver with Master: 4893a8ea566e45eb8315ade7291a5650 (127.26.250.129:37751)
I20260812 03:08:18.361538 31432 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.129@KRBTEST.COM'} at 127.26.250.129:47231
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
I20260812 03:08:18.374044 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:0
--local_ip_for_outbound_sockets=127.26.250.130
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39453
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:41299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:18.494926 31638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:18.495258 31638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:18.495353 31638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:18.499593 31638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:18.499698 31638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:18.499797 31638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:08:18.505378 31638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:41299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:18.506618 31638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:18.507714 31638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:18.518025 31644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:18.518136 31645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:18.518040 31647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:18.518254 31638 server_base.cc:1034] running on GCE node
I20260812 03:08:18.518879 31638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:18.519511 31638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:18.520756 31638 hybrid_clock.cc:648] HybridClock initialized: now 1786504098520682 us; error 30 us; skew 500 ppm
Aug 12 03:08:18 dist-test-slave-zpfg krb5kdc[31384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504098, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:18.524057 31638 init.cc:377] Logged in from keytab as kudu/127.26.250.130@KRBTEST.COM (short username kudu)
I20260812 03:08:18.525183 31638 webserver.cc:460] Webserver started at http://127.26.250.130:42699/ using document root <none> and password file <none>
I20260812 03:08:18.525837 31638 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:18.525906 31638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:18.526189 31638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:18.528520 31638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-1/data/instance:
uuid: "c43d2f5566f24c41a3278f974af4efd7"
format_stamp: "Formatted at 2026-08-12 03:08:18 on dist-test-slave-zpfg"
server_key: "28bc686b9acccae0549f48a5fd737d21"
server_key_iv: "6ded9f23622933d63b4eb9e9c5079f46"
server_key_version: "encryptionkey@0"
I20260812 03:08:18.529127 31638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance:
uuid: "c43d2f5566f24c41a3278f974af4efd7"
format_stamp: "Formatted at 2026-08-12 03:08:18 on dist-test-slave-zpfg"
server_key: "28bc686b9acccae0549f48a5fd737d21"
server_key_iv: "6ded9f23622933d63b4eb9e9c5079f46"
server_key_version: "encryptionkey@0"
I20260812 03:08:18.533504 31638 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:08:18.536391 31654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:18.537765 31638 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:08:18.537906 31638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "c43d2f5566f24c41a3278f974af4efd7"
format_stamp: "Formatted at 2026-08-12 03:08:18 on dist-test-slave-zpfg"
server_key: "28bc686b9acccae0549f48a5fd737d21"
server_key_iv: "6ded9f23622933d63b4eb9e9c5079f46"
server_key_version: "encryptionkey@0"
I20260812 03:08:18.538024 31638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:18.567575 31638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:18.571270 31638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:18.571514 31638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:18.572243 31638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:18.573186 31638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:18.573236 31638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:18.573278 31638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:18.573318 31638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:18.584076 31638 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:46505
I20260812 03:08:18.584100 31773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:46505 every 8 connection(s)
I20260812 03:08:18.585088 31638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:08:18.592736 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 31638
I20260812 03:08:18.592881 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance
I20260812 03:08:18.593194 27626 external_mini_cluster.cc:1428] Setting key 02964241b0e6e0ca7eb5628fd759570b
Aug 12 03:08:18 dist-test-slave-zpfg krb5kdc[31384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504098, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:18.600121 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:18.587131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:38577 (local address 127.26.250.190:39453)
0812 03:08:18.587278 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:18.587282 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:18.588180 (+ 898us) server_negotiation.cc:407] Connection header received
0812 03:08:18.589624 (+ 1444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:18.589628 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:18.589684 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:18.589794 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:18.591108 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:18.591646 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:18.592829 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:18.593148 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:18.593266 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:18.596217 (+ 2951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:18.596221 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:18.596223 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:18.596247 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:18.597766 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:18.598388 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:18.598412 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:18.598415 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:18.598505 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:18.598883 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:18.598886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:18.598888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:18.599095 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:18.599209 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:18.599800 (+ 591us) server_negotiation.cc:299] Negotiation successful
0812 03:08:18.599948 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:08:18.601044 31777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:18.587527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39453 (local address 127.26.250.130:38577)
0812 03:08:18.588020 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:18.588070 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:18.589407 (+ 1337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:18.589977 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:18.589984 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:18.590431 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:18.590955 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:18.590969 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:18.591790 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:18.591794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:18.592693 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:18.592702 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:18.593320 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:18.593325 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:18.593502 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:18.594211 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:18.594229 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:18.596001 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:18.597918 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:18.597925 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:18.597944 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:18.598239 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:18.598608 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:18.598611 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:18.598613 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:18.598778 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:18.599211 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:18.599221 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:18.599514 (+ 293us) client_negotiation.cc:769] Sending connection context
0812 03:08:18.599754 (+ 240us) client_negotiation.cc:240] Negotiation successful
0812 03:08:18.600134 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":134,"threads_started":1}
I20260812 03:08:18.602654 31774 heartbeater.cc:347] Connected to a master server at 127.26.250.190:39453
I20260812 03:08:18.602963 31774 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:18.603665 31774 heartbeater.cc:511] Master 127.26.250.190:39453 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.26.250.131@KRBTEST.COM; defaulting to no policy
I20260812 03:08:18.605053 31432 ts_manager.cc:194] Registered new tserver with Master: c43d2f5566f24c41a3278f974af4efd7 (127.26.250.130:46505)
I20260812 03:08:18.605721 31432 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.130@KRBTEST.COM'} at 127.26.250.130:38577
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
I20260812 03:08:18.626906 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:0
--local_ip_for_outbound_sockets=127.26.250.131
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39453
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:41299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:18.747649 31780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:18.747984 31780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:18.748060 31780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:18.752216 31780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:18.752323 31780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:18.752424 31780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:08:18.758080 31780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:41299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:18.759285 31780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:18.760394 31780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:18.770888 31788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:18.770903 31786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:18.770906 31785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:18.771201 31780 server_base.cc:1034] running on GCE node
I20260812 03:08:18.771850 31780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:18.772603 31780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:18.773819 31780 hybrid_clock.cc:648] HybridClock initialized: now 1786504098773772 us; error 62 us; skew 500 ppm
Aug 12 03:08:18 dist-test-slave-zpfg krb5kdc[31384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504098, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:18.778244 31780 init.cc:377] Logged in from keytab as kudu/127.26.250.131@KRBTEST.COM (short username kudu)
I20260812 03:08:18.779569 31780 webserver.cc:460] Webserver started at http://127.26.250.131:42731/ using document root <none> and password file <none>
I20260812 03:08:18.780355 31780 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:18.780467 31780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:18.780707 31780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:18.783071 31780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-2/data/instance:
uuid: "795fb0442df34b0fa3f0c4d700bf8d28"
format_stamp: "Formatted at 2026-08-12 03:08:18 on dist-test-slave-zpfg"
server_key: "311e510e33b937555b011a22529d62e0"
server_key_iv: "93740812a554cb6102c0561608d1b4a2"
server_key_version: "encryptionkey@0"
I20260812 03:08:18.783686 31780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance:
uuid: "795fb0442df34b0fa3f0c4d700bf8d28"
format_stamp: "Formatted at 2026-08-12 03:08:18 on dist-test-slave-zpfg"
server_key: "311e510e33b937555b011a22529d62e0"
server_key_iv: "93740812a554cb6102c0561608d1b4a2"
server_key_version: "encryptionkey@0"
I20260812 03:08:18.788175 31780 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:08:18.791291 31796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:18.792866 31780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:08:18.793031 31780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "795fb0442df34b0fa3f0c4d700bf8d28"
format_stamp: "Formatted at 2026-08-12 03:08:18 on dist-test-slave-zpfg"
server_key: "311e510e33b937555b011a22529d62e0"
server_key_iv: "93740812a554cb6102c0561608d1b4a2"
server_key_version: "encryptionkey@0"
I20260812 03:08:18.793165 31780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:18.822974 31780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:18.826690 31780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:18.826925 31780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:18.827594 31780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:18.828480 31780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:18.828526 31780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:18.828588 31780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:18.828622 31780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:18.838322 31780 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:32811
I20260812 03:08:18.838397 31920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:32811 every 8 connection(s)
I20260812 03:08:18.839349 31780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:08:18.845680 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 31780
I20260812 03:08:18.845855 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance
I20260812 03:08:18.846153 27626 external_mini_cluster.cc:1428] Setting key 1b347b2419931d7f712b300878b748ca
Aug 12 03:08:18 dist-test-slave-zpfg krb5kdc[31384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504098, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:18.854756 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:18.841300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:46959 (local address 127.26.250.190:39453)
0812 03:08:18.841455 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:18.841459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:18.842657 (+ 1198us) server_negotiation.cc:407] Connection header received
0812 03:08:18.844199 (+ 1542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:18.844203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:18.844264 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:18.844419 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:18.845841 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:18.846554 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:18.848002 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:18.848325 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:18.848445 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:18.851146 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:18.851150 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:18.851196 (+ 46us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:18.851225 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:18.852770 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:18.853288 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:18.853290 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:18.853292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:18.853366 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:18.853691 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:18.853698 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:18.853699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:18.853886 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:18.853966 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:18.854507 (+ 541us) server_negotiation.cc:299] Negotiation successful
0812 03:08:18.854619 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 03:08:18.855698 31923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:18.841924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39453 (local address 127.26.250.131:46959)
0812 03:08:18.842504 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:18.842550 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:18.843972 (+ 1422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:18.844588 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:18.844595 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:18.844975 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:18.845613 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:18.845638 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:18.846698 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:18.846705 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:18.847889 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:18.847900 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:18.848508 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:18.848514 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:18.848682 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:18.849343 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:18.849360 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:18.850971 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:18.852892 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:18.852896 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:18.852909 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:18.853166 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:18.853452 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:18.853455 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:18.853456 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:18.853599 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:18.853978 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:18.853984 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:18.854234 (+ 250us) client_negotiation.cc:769] Sending connection context
0812 03:08:18.854464 (+ 230us) client_negotiation.cc:240] Negotiation successful
0812 03:08:18.854749 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":133,"threads_started":1}
I20260812 03:08:18.857051 31921 heartbeater.cc:347] Connected to a master server at 127.26.250.190:39453
I20260812 03:08:18.857309 31921 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:18.857872 31921 heartbeater.cc:511] Master 127.26.250.190:39453 requested a full tablet report, sending...
I20260812 03:08:18.859054 31432 ts_manager.cc:194] Registered new tserver with Master: 795fb0442df34b0fa3f0c4d700bf8d28 (127.26.250.131:32811)
I20260812 03:08:18.859906 31432 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.131@KRBTEST.COM'} at 127.26.250.131:46959
I20260812 03:08:18.860641 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 03:08:18 dist-test-slave-zpfg krb5kdc[31384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:08:18 dist-test-slave-zpfg krb5kdc[31384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.129@KRBTEST.COM
I20260812 03:08:18.888788 31928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:18.878766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48124 (local address 127.26.250.129:37751)
0812 03:08:18.879077 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:18.879086 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:18.879118 (+ 32us) server_negotiation.cc:407] Connection header received
0812 03:08:18.879185 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:18.879190 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:18.879443 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:18.879582 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:18.880309 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:18.881110 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:18.882027 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:18.882530 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:18.882621 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:18.885046 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:18.885058 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:18.885073 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:18.885099 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:18.887003 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:18.887470 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:18.887475 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:18.887477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:18.887563 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:18.887868 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:18.887870 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:18.887872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:18.888132 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:18.888295 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:18.888506 (+ 211us) server_negotiation.cc:299] Negotiation successful
0812 03:08:18.888642 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":123,"threads_started":1}
Aug 12 03:08:18 dist-test-slave-zpfg krb5kdc[31384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:18.902887 31472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:18.891999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44416 (local address 127.26.250.190:39453)
0812 03:08:18.892147 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:18.892154 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:18.892198 (+ 44us) server_negotiation.cc:407] Connection header received
0812 03:08:18.892413 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:18.892417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:18.892469 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:18.892559 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:18.893201 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:18.893726 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:18.894727 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:18.895127 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:18.895254 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:18.898409 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:18.898416 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:18.898420 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:18.898485 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:18.900562 (+ 2077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:18.901176 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:18.901180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:18.901182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:18.901295 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:18.901620 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:18.901624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:18.901626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:18.902004 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:18.902158 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:18.902514 (+ 356us) server_negotiation.cc:299] Negotiation successful
0812 03:08:18.902681 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260812 03:08:18.903564 31431 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44416
I20260812 03:08:18.904353 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 31496
I20260812 03:08:18.916741 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 31638
I20260812 03:08:18.928194 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 31780
I20260812 03:08:18.942808 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 31400
2026-08-12T03:08:18Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2347 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:08:19 dist-test-slave-zpfg krb5kdc[31933](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:08:19 dist-test-slave-zpfg krb5kdc[31933](info): set up 2 sockets
Aug 12 03:08:19 dist-test-slave-zpfg krb5kdc[31933](info): commencing operation
krb5kdc: starting...
W20260812 03:08:20.055641 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:08:20 dist-test-slave-zpfg krb5kdc[31933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:08:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:08:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:08:20.198765 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:36063
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:37311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:36063
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:20.321288 31946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:20.321588 31946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:20.321633 31946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:20.325737 31946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:08:20.325811 31946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:20.325837 31946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:20.325894 31946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:08:20.325923 31946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:08:20.331483 31946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:37311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:36063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:36063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:20.332593 31946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:20.333634 31946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:20.342769 31952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:20.342774 31954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:20.342769 31951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:20.343312 31946 server_base.cc:1034] running on GCE node
I20260812 03:08:20.343884 31946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:20.345036 31946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:20.346210 31946 hybrid_clock.cc:648] HybridClock initialized: now 1786504100346182 us; error 44 us; skew 500 ppm
Aug 12 03:08:20 dist-test-slave-zpfg krb5kdc[31933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504100, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:20.349336 31946 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:08:20.350132 31946 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:20.350204 31946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:20.350468 31946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:20.352675 31946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "89f216816a5b4254af638d6255c2ad25"
format_stamp: "Formatted at 2026-08-12 03:08:20 on dist-test-slave-zpfg"
server_key: "096f37dd54b47bdf342fad732e8e7821"
server_key_iv: "c9a454846cdb5f449caefdfd2541bcd9"
server_key_version: "encryptionkey@0"
I20260812 03:08:20.353252 31946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "89f216816a5b4254af638d6255c2ad25"
format_stamp: "Formatted at 2026-08-12 03:08:20 on dist-test-slave-zpfg"
server_key: "096f37dd54b47bdf342fad732e8e7821"
server_key_iv: "c9a454846cdb5f449caefdfd2541bcd9"
server_key_version: "encryptionkey@0"
I20260812 03:08:20.357366 31946 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 03:08:20.359961 31961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:20.361284 31946 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 03:08:20.361423 31946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "89f216816a5b4254af638d6255c2ad25"
format_stamp: "Formatted at 2026-08-12 03:08:20 on dist-test-slave-zpfg"
server_key: "096f37dd54b47bdf342fad732e8e7821"
server_key_iv: "c9a454846cdb5f449caefdfd2541bcd9"
server_key_version: "encryptionkey@0"
I20260812 03:08:20.361537 31946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:20.384250 31946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:20.387971 31946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:20.388198 31946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:20.395905 31946 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:36063
I20260812 03:08:20.396230 32018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:36063 every 8 connection(s)
I20260812 03:08:20.396713 31946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:08:20.397454 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 31946
I20260812 03:08:20.397595 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:08:20.397886 27626 external_mini_cluster.cc:1428] Setting key 23451df77e9e51f51e05875904a4520b
I20260812 03:08:20.400207 32019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:20.405967 32019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89f216816a5b4254af638d6255c2ad25: Bootstrap starting.
Aug 12 03:08:20 dist-test-slave-zpfg krb5kdc[31933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504100, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:20.408507 32019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89f216816a5b4254af638d6255c2ad25: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:20.409358 32019 log.cc:824] T 00000000000000000000000000000000 P 89f216816a5b4254af638d6255c2ad25: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:20.411372 32019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89f216816a5b4254af638d6255c2ad25: No bootstrap required, opened a new log
I20260812 03:08:20.412562 32022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:20.399443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56328 (local address 127.26.250.190:36063)
0812 03:08:20.399925 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:20.399941 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:20.399978 (+ 37us) server_negotiation.cc:407] Connection header received
0812 03:08:20.400700 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:20.400719 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:20.401065 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:20.401489 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:20.402283 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:20.403398 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:20.404298 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:20.404789 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:20.404964 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:20.407641 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:20.407647 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:20.407661 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:20.407691 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:20.409643 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:20.410127 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:20.410132 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:20.410137 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:20.410254 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:20.410582 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:20.410586 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:20.410588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:20.411005 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:20.411225 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:20.411520 (+ 295us) server_negotiation.cc:299] Negotiation successful
0812 03:08:20.411831 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":157,"threads_started":1}
I20260812 03:08:20.415172 32019 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89f216816a5b4254af638d6255c2ad25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f216816a5b4254af638d6255c2ad25" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 36063 } }
I20260812 03:08:20.415426 32019 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89f216816a5b4254af638d6255c2ad25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:20.415510 32019 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89f216816a5b4254af638d6255c2ad25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f216816a5b4254af638d6255c2ad25, State: Initialized, Role: FOLLOWER
I20260812 03:08:20.415983 32019 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89f216816a5b4254af638d6255c2ad25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f216816a5b4254af638d6255c2ad25" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 36063 } }
I20260812 03:08:20.416157 32019 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89f216816a5b4254af638d6255c2ad25 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:08:20.416246 32019 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89f216816a5b4254af638d6255c2ad25 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:08:20.416349 32019 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89f216816a5b4254af638d6255c2ad25 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:20.417407 32019 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89f216816a5b4254af638d6255c2ad25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f216816a5b4254af638d6255c2ad25" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 36063 } }
I20260812 03:08:20.417754 32019 leader_election.cc:302] T 00000000000000000000000000000000 P 89f216816a5b4254af638d6255c2ad25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89f216816a5b4254af638d6255c2ad25; no voters:
I20260812 03:08:20.418143 32019 leader_election.cc:288] T 00000000000000000000000000000000 P 89f216816a5b4254af638d6255c2ad25 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:08:20.418401 32025 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89f216816a5b4254af638d6255c2ad25 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:20.418665 32025 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89f216816a5b4254af638d6255c2ad25 [term 1 LEADER]: Becoming Leader. State: Replica: 89f216816a5b4254af638d6255c2ad25, State: Running, Role: LEADER
I20260812 03:08:20.418977 32025 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89f216816a5b4254af638d6255c2ad25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f216816a5b4254af638d6255c2ad25" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 36063 } }
I20260812 03:08:20.419493 32019 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89f216816a5b4254af638d6255c2ad25 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:08:20.420753 32026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89f216816a5b4254af638d6255c2ad25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89f216816a5b4254af638d6255c2ad25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f216816a5b4254af638d6255c2ad25" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 36063 } } }
I20260812 03:08:20.420866 32026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89f216816a5b4254af638d6255c2ad25 [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:20.421096 32027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89f216816a5b4254af638d6255c2ad25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89f216816a5b4254af638d6255c2ad25. Latest consensus state: current_term: 1 leader_uuid: "89f216816a5b4254af638d6255c2ad25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f216816a5b4254af638d6255c2ad25" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 36063 } } }
I20260812 03:08:20.421198 32027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89f216816a5b4254af638d6255c2ad25 [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:20.421545 32034 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:08:20.424438 32034 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:08:20.430142 32034 catalog_manager.cc:1347] Generated new cluster ID: 315624f1550b420a82dae538acedc931
I20260812 03:08:20.430225 32034 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:08:20.450805 32034 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:08:20.451942 32034 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:08:20.457595 32034 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89f216816a5b4254af638d6255c2ad25: Generated new TSK 0
I20260812 03:08:20.458607 32034 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.250.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
I20260812 03:08:20.502727 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:0
--local_ip_for_outbound_sockets=127.26.250.129
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:36063
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:37311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:20.625542 32046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:20.625885 32046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:20.625970 32046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:20.630191 32046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:20.630404 32046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:20.630544 32046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:08:20.636178 32046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:37311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:36063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:20.637351 32046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:20.638523 32046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:20.648676 32051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:20.648676 32052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:20.648712 32056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:20.649111 32046 server_base.cc:1034] running on GCE node
I20260812 03:08:20.649595 32046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:20.650445 32046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:20.651654 32046 hybrid_clock.cc:648] HybridClock initialized: now 1786504100651609 us; error 52 us; skew 500 ppm
Aug 12 03:08:20 dist-test-slave-zpfg krb5kdc[31933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504100, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:20.654747 32046 init.cc:377] Logged in from keytab as kudu/127.26.250.129@KRBTEST.COM (short username kudu)
I20260812 03:08:20.655591 32046 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:20.655690 32046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:20.655961 32046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:20.658252 32046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-0/data/instance:
uuid: "b11a569d8f3043709142c20a6c283287"
format_stamp: "Formatted at 2026-08-12 03:08:20 on dist-test-slave-zpfg"
server_key: "527ee55dfe9e64e59823ef68130c2939"
server_key_iv: "afaa625689e40019e1d1bc424c506afb"
server_key_version: "encryptionkey@0"
I20260812 03:08:20.658923 32046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance:
uuid: "b11a569d8f3043709142c20a6c283287"
format_stamp: "Formatted at 2026-08-12 03:08:20 on dist-test-slave-zpfg"
server_key: "527ee55dfe9e64e59823ef68130c2939"
server_key_iv: "afaa625689e40019e1d1bc424c506afb"
server_key_version: "encryptionkey@0"
I20260812 03:08:20.662971 32046 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 03:08:20.665876 32063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:20.667297 32046 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:08:20.667446 32046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "b11a569d8f3043709142c20a6c283287"
format_stamp: "Formatted at 2026-08-12 03:08:20 on dist-test-slave-zpfg"
server_key: "527ee55dfe9e64e59823ef68130c2939"
server_key_iv: "afaa625689e40019e1d1bc424c506afb"
server_key_version: "encryptionkey@0"
I20260812 03:08:20.667570 32046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:20.688241 32046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:20.692310 32046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:20.692561 32046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:20.693264 32046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:20.694265 32046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:20.694344 32046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:20.694437 32046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:20.694485 32046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:20.705690 32046 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:35715
I20260812 03:08:20.705714 32183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:35715 every 8 connection(s)
I20260812 03:08:20.706507 32046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:08:20.711418 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 32046
I20260812 03:08:20.711526 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance
I20260812 03:08:20.711849 27626 external_mini_cluster.cc:1428] Setting key 7854cf77d4b44ecfb209c54239260313
Aug 12 03:08:20 dist-test-slave-zpfg krb5kdc[31933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504100, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:20.722503 32022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:20.708508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:55475 (local address 127.26.250.190:36063)
0812 03:08:20.708652 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:20.708656 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:20.709554 (+ 898us) server_negotiation.cc:407] Connection header received
0812 03:08:20.711036 (+ 1482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:20.711040 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:20.711102 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:20.711194 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:20.712714 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:20.713376 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:20.714631 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:20.715046 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:20.715182 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:20.717975 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:20.717979 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:20.717982 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:20.718018 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:20.719763 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:20.720551 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:20.720555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:20.720557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:20.720650 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:20.721127 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:20.721130 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:20.721131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:20.721328 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:20.721417 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:20.722167 (+ 750us) server_negotiation.cc:299] Negotiation successful
0812 03:08:20.722296 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:08:20.723819 32186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:20.708902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:36063 (local address 127.26.250.129:55475)
0812 03:08:20.709394 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:20.709440 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:20.710786 (+ 1346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:20.711380 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:20.711388 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:20.711867 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:20.712535 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:20.712554 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:20.713526 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:20.713529 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:20.714498 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:20.714508 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:20.715215 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:20.715218 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:20.715375 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:20.716054 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:20.716071 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:20.717744 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:20.719942 (+ 2198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:20.719952 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:20.719974 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:20.720410 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:20.720763 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:20.720769 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:20.720772 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:20.721014 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:20.721452 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:20.721463 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:20.721864 (+ 401us) client_negotiation.cc:769] Sending connection context
0812 03:08:20.722104 (+ 240us) client_negotiation.cc:240] Negotiation successful
0812 03:08:20.722558 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":128,"threads_started":1}
WARNING: no policy specified for kudu/127.26.250.130@KRBTEST.COM; defaulting to no policy
I20260812 03:08:20.725194 32184 heartbeater.cc:347] Connected to a master server at 127.26.250.190:36063
I20260812 03:08:20.725492 32184 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:20.725991 32184 heartbeater.cc:511] Master 127.26.250.190:36063 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.130@KRBTEST.COM" created.
I20260812 03:08:20.727816 31979 ts_manager.cc:194] Registered new tserver with Master: b11a569d8f3043709142c20a6c283287 (127.26.250.129:35715)
I20260812 03:08:20.729465 31979 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.129@KRBTEST.COM'} at 127.26.250.129:55475
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
I20260812 03:08:20.746629 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:0
--local_ip_for_outbound_sockets=127.26.250.130
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:36063
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:37311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:20.878099 32190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:20.878440 32190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:20.878549 32190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:20.884111 32190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:20.884228 32190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:20.884353 32190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:08:20.890403 32190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:37311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:36063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:20.891669 32190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:20.892824 32190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:20.903887 32196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:20.903914 32199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:20.903914 32197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:20.904136 32190 server_base.cc:1034] running on GCE node
I20260812 03:08:20.904713 32190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:20.905436 32190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:20.906570 32190 hybrid_clock.cc:648] HybridClock initialized: now 1786504100906527 us; error 56 us; skew 500 ppm
Aug 12 03:08:20 dist-test-slave-zpfg krb5kdc[31933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504100, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:20.909848 32190 init.cc:377] Logged in from keytab as kudu/127.26.250.130@KRBTEST.COM (short username kudu)
I20260812 03:08:20.910789 32190 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:20.910904 32190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:20.911144 32190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:20.913484 32190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-1/data/instance:
uuid: "f0f3cfe9fc204c4c9eff0d527ea7e980"
format_stamp: "Formatted at 2026-08-12 03:08:20 on dist-test-slave-zpfg"
server_key: "d5136f6f110be8f0ecd2fe115483f636"
server_key_iv: "8ebc82cce8c3723ecd5604359af5b006"
server_key_version: "encryptionkey@0"
I20260812 03:08:20.914166 32190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance:
uuid: "f0f3cfe9fc204c4c9eff0d527ea7e980"
format_stamp: "Formatted at 2026-08-12 03:08:20 on dist-test-slave-zpfg"
server_key: "d5136f6f110be8f0ecd2fe115483f636"
server_key_iv: "8ebc82cce8c3723ecd5604359af5b006"
server_key_version: "encryptionkey@0"
I20260812 03:08:20.918506 32190 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 03:08:20.921722 32205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:20.923347 32190 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:08:20.923519 32190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "f0f3cfe9fc204c4c9eff0d527ea7e980"
format_stamp: "Formatted at 2026-08-12 03:08:20 on dist-test-slave-zpfg"
server_key: "d5136f6f110be8f0ecd2fe115483f636"
server_key_iv: "8ebc82cce8c3723ecd5604359af5b006"
server_key_version: "encryptionkey@0"
I20260812 03:08:20.923658 32190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:20.940239 32190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:20.944026 32190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:20.944291 32190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:20.945012 32190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:20.946009 32190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:20.946095 32190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:20.946172 32190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:20.946226 32190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:20.956895 32190 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:42333
I20260812 03:08:20.956947 32324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:42333 every 8 connection(s)
I20260812 03:08:20.957726 32190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:08:20.965128 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 32190
I20260812 03:08:20.965255 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance
I20260812 03:08:20.965565 27626 external_mini_cluster.cc:1428] Setting key ff3945453b21c2dac6f8d43b7ea9dc1c
Aug 12 03:08:20 dist-test-slave-zpfg krb5kdc[31933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504100, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:20.973383 32022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:20.959684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:45827 (local address 127.26.250.190:36063)
0812 03:08:20.959829 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:20.959833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:20.960753 (+ 920us) server_negotiation.cc:407] Connection header received
0812 03:08:20.962046 (+ 1293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:20.962049 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:20.962098 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:20.962180 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:20.963607 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:20.964316 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:20.965768 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:20.966131 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:20.966245 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:20.969417 (+ 3172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:20.969420 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:20.969422 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:20.969448 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:20.971028 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:20.971609 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:20.971617 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:20.971619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:20.971693 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:20.972158 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:20.972165 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:20.972170 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:20.972376 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:20.972485 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:20.973070 (+ 585us) server_negotiation.cc:299] Negotiation successful
0812 03:08:20.973207 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:08:20.974323 32327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:20.960089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:36063 (local address 127.26.250.130:45827)
0812 03:08:20.960591 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:20.960636 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:20.961831 (+ 1195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:20.962466 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:20.962474 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:20.962902 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:20.963434 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:20.963448 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:20.964463 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:20.964465 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:20.965606 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:20.965617 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:20.966407 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:20.966413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:20.966603 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:20.967360 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:20.967383 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:20.969173 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:20.971161 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:20.971167 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:20.971186 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:20.971489 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:20.971788 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:20.971792 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:20.971794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:20.971972 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:20.972496 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:20.972505 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:20.972794 (+ 289us) client_negotiation.cc:769] Sending connection context
0812 03:08:20.973014 (+ 220us) client_negotiation.cc:240] Negotiation successful
0812 03:08:20.973454 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":106,"threads_started":1}
I20260812 03:08:20.975642 32325 heartbeater.cc:347] Connected to a master server at 127.26.250.190:36063
I20260812 03:08:20.975919 32325 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:20.976439 32325 heartbeater.cc:511] Master 127.26.250.190:36063 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.26.250.131@KRBTEST.COM; defaulting to no policy
I20260812 03:08:20.977538 31979 ts_manager.cc:194] Registered new tserver with Master: f0f3cfe9fc204c4c9eff0d527ea7e980 (127.26.250.130:42333)
I20260812 03:08:20.978241 31979 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.130@KRBTEST.COM'} at 127.26.250.130:45827
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
I20260812 03:08:20.999197 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:0
--local_ip_for_outbound_sockets=127.26.250.131
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:36063
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:37311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:21.125383 32330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:21.125684 32330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:21.125782 32330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:21.130424 32330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:21.130506 32330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:21.130573 32330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:08:21.136456 32330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:37311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:36063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:21.137610 32330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:21.138758 32330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:21.149187 32335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:21.149255 32336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:21.149255 32338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:21.149735 32330 server_base.cc:1034] running on GCE node
I20260812 03:08:21.150245 32330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:21.151036 32330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:21.152256 32330 hybrid_clock.cc:648] HybridClock initialized: now 1786504101152186 us; error 83 us; skew 500 ppm
Aug 12 03:08:21 dist-test-slave-zpfg krb5kdc[31933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504101, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:21.156041 32330 init.cc:377] Logged in from keytab as kudu/127.26.250.131@KRBTEST.COM (short username kudu)
I20260812 03:08:21.157150 32330 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:21.157264 32330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:21.157536 32330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:21.159984 32330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-2/data/instance:
uuid: "2c310c6a38754961a8795b0efbad0eab"
format_stamp: "Formatted at 2026-08-12 03:08:21 on dist-test-slave-zpfg"
server_key: "18f408899f9bfb1c77fcc85b70b32e08"
server_key_iv: "0407b0a18da151b9b220948be9a52b51"
server_key_version: "encryptionkey@0"
I20260812 03:08:21.160653 32330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance:
uuid: "2c310c6a38754961a8795b0efbad0eab"
format_stamp: "Formatted at 2026-08-12 03:08:21 on dist-test-slave-zpfg"
server_key: "18f408899f9bfb1c77fcc85b70b32e08"
server_key_iv: "0407b0a18da151b9b220948be9a52b51"
server_key_version: "encryptionkey@0"
I20260812 03:08:21.165498 32330 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:08:21.168713 32345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:21.170188 32330 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 03:08:21.170395 32330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "2c310c6a38754961a8795b0efbad0eab"
format_stamp: "Formatted at 2026-08-12 03:08:21 on dist-test-slave-zpfg"
server_key: "18f408899f9bfb1c77fcc85b70b32e08"
server_key_iv: "0407b0a18da151b9b220948be9a52b51"
server_key_version: "encryptionkey@0"
I20260812 03:08:21.170542 32330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:21.184499 32330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:21.188189 32330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:21.191509 32330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:21.192324 32330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:21.193382 32330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:21.193460 32330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:21.193542 32330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:21.193598 32330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:21.204197 32330 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:41645
I20260812 03:08:21.204303 32462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:41645 every 8 connection(s)
I20260812 03:08:21.204955 32330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:08:21.207474 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 32330
I20260812 03:08:21.207619 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance
I20260812 03:08:21.207967 27626 external_mini_cluster.cc:1428] Setting key 32de22a3b5b1d1365dd6e2715a990422
Aug 12 03:08:21 dist-test-slave-zpfg krb5kdc[31933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504101, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:21.220609 32022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:21.207092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:48465 (local address 127.26.250.190:36063)
0812 03:08:21.207251 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:21.207254 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:21.208407 (+ 1153us) server_negotiation.cc:407] Connection header received
0812 03:08:21.210206 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:21.210209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:21.210265 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:21.210374 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:21.211599 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.212139 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.213357 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.213587 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.213700 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:21.216469 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:21.216478 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:21.216480 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:21.216507 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:21.218168 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:21.218809 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:21.218813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:21.218815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:21.218902 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:21.219273 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:21.219277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:21.219279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:21.219594 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:21.219704 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:21.220282 (+ 578us) server_negotiation.cc:299] Negotiation successful
0812 03:08:21.220433 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 03:08:21.221837 32465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:21.207627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:36063 (local address 127.26.250.131:48465)
0812 03:08:21.208236 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:21.208293 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:21.209965 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:21.210549 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:21.210556 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:21.210933 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:21.211436 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:21.211450 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.212274 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.212278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:21.213210 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:21.213220 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.213758 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.213762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:21.213953 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:21.214634 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:21.214652 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:21.216232 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:21.218321 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:21.218328 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:21.218396 (+ 68us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:21.218695 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:21.219000 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:21.219003 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:21.219006 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:21.219165 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:21.219704 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:21.219712 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:21.220004 (+ 292us) client_negotiation.cc:769] Sending connection context
0812 03:08:21.220226 (+ 222us) client_negotiation.cc:240] Negotiation successful
0812 03:08:21.220679 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":169,"threads_started":1}
I20260812 03:08:21.223151 32463 heartbeater.cc:347] Connected to a master server at 127.26.250.190:36063
I20260812 03:08:21.223440 32463 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:21.223953 32463 heartbeater.cc:511] Master 127.26.250.190:36063 requested a full tablet report, sending...
I20260812 03:08:21.225097 31979 ts_manager.cc:194] Registered new tserver with Master: 2c310c6a38754961a8795b0efbad0eab (127.26.250.131:41645)
I20260812 03:08:21.225749 31979 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.131@KRBTEST.COM'} at 127.26.250.131:48465
I20260812 03:08:21.235162 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:08:21.246850 32022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:21.237609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56330 (local address 127.26.250.190:36063)
0812 03:08:21.237931 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:21.237936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:21.238110 (+ 174us) server_negotiation.cc:407] Connection header received
0812 03:08:21.238320 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:21.238323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:21.238426 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:21.238521 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:21.239414 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.240004 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.241024 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.241273 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.241400 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:21.243048 (+ 1648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:21.243052 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:21.243054 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:21.243084 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:21.244749 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:21.245270 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:21.245304 (+ 34us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:21.245308 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:21.245423 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:21.245754 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:21.245757 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:21.245759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:21.246044 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:21.246162 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:21.246491 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 03:08:21.246612 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204}
I20260812 03:08:21.250200 31979 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:08:21.253731 31979 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:08:21.270910 32478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:21.264982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54484 (local address 127.26.250.131:41645)
0812 03:08:21.265732 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:21.265743 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:21.265765 (+ 22us) server_negotiation.cc:407] Connection header received
0812 03:08:21.265890 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:21.265899 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:21.266089 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:21.266403 (+ 314us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:21.266589 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.267529 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.269340 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.270094 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.270251 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:21.270445 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:21.270612 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 03:08:21.270727 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":158,"threads_started":1}
I20260812 03:08:21.270907 32475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:21.264079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:41645 (local address 127.0.0.1:54484)
0812 03:08:21.265087 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:21.265149 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:21.265306 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:21.266259 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:21.266262 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:21.266277 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:21.266437 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:21.266446 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.267664 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.267671 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:21.269188 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:21.269199 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.270236 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.270240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:21.270383 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:21.270427 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 03:08:21.270576 (+ 149us) client_negotiation.cc:240] Negotiation successful
0812 03:08:21.270719 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":895,"mutex_wait_us":235,"spinlock_wait_cycles":2432,"thread_start_us":156,"threads_started":1}
I20260812 03:08:21.271384 32474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:21.264440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:35715 (local address 127.0.0.1:59016)
0812 03:08:21.265276 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:21.265289 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:21.265379 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:21.267009 (+ 1630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:21.267012 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:21.267031 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:21.267120 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:21.267137 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.268432 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.268436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:21.269624 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:21.269635 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.270922 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.270926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:21.271058 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:21.271084 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 03:08:21.271136 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 03:08:21.271234 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"mutex_wait_us":12,"thread_start_us":99,"threads_started":1}
I20260812 03:08:21.271596 32476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:21.264242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33144 (local address 127.26.250.130:42333)
0812 03:08:21.265485 (+ 1243us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:21.265526 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:21.265556 (+ 30us) server_negotiation.cc:407] Connection header received
0812 03:08:21.265639 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:21.265647 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:21.265901 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:21.266069 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:21.266680 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.267877 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.269581 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.270538 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.270707 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:21.270865 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:21.271299 (+ 434us) server_negotiation.cc:299] Negotiation successful
0812 03:08:21.271411 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":795,"thread_start_us":167,"threads_started":1}
I20260812 03:08:21.272140 32477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:21.264079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:42333 (local address 127.0.0.1:33144)
0812 03:08:21.265086 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:21.265147 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:21.265320 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:21.266073 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:21.266088 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:21.266113 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:21.266552 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:21.266559 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.268030 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.268034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:21.269426 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:21.269437 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.270922 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.270926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:21.271149 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:21.271166 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 03:08:21.271935 (+ 769us) client_negotiation.cc:240] Negotiation successful
0812 03:08:21.272035 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":803,"mutex_wait_us":71,"spinlock_wait_cycles":7808,"thread_start_us":78,"threads_started":1}
I20260812 03:08:21.272141 32479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:21.265431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59016 (local address 127.26.250.129:35715)
0812 03:08:21.266499 (+ 1068us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:21.266505 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:21.266525 (+ 20us) server_negotiation.cc:407] Connection header received
0812 03:08:21.266644 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:21.266649 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:21.266852 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:21.267006 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:21.267265 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.268254 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.269767 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.270770 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.271713 (+ 943us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:21.271859 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:21.271910 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 03:08:21.271999 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":973,"thread_start_us":85,"threads_started":1}
I20260812 03:08:21.275032 32255 tablet_service.cc:1467] Processing CreateTablet for tablet 6e37f44a332141e48659a51b96661236 (DEFAULT_TABLE table=test-table [id=f66dbc8b8a4d4a269f16868b445c9b23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:21.275027 32394 tablet_service.cc:1467] Processing CreateTablet for tablet 6e37f44a332141e48659a51b96661236 (DEFAULT_TABLE table=test-table [id=f66dbc8b8a4d4a269f16868b445c9b23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:21.275017 32114 tablet_service.cc:1467] Processing CreateTablet for tablet 6e37f44a332141e48659a51b96661236 (DEFAULT_TABLE table=test-table [id=f66dbc8b8a4d4a269f16868b445c9b23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:21.276175 32114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e37f44a332141e48659a51b96661236. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:21.276196 32255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e37f44a332141e48659a51b96661236. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:21.276225 32394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e37f44a332141e48659a51b96661236. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:21.283113 32481 tablet_bootstrap.cc:492] T 6e37f44a332141e48659a51b96661236 P b11a569d8f3043709142c20a6c283287: Bootstrap starting.
I20260812 03:08:21.284880 32482 tablet_bootstrap.cc:492] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980: Bootstrap starting.
I20260812 03:08:21.285619 32481 tablet_bootstrap.cc:654] T 6e37f44a332141e48659a51b96661236 P b11a569d8f3043709142c20a6c283287: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:21.286518 32481 log.cc:824] T 6e37f44a332141e48659a51b96661236 P b11a569d8f3043709142c20a6c283287: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:21.287005 32482 tablet_bootstrap.cc:654] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:21.287288 32483 tablet_bootstrap.cc:492] T 6e37f44a332141e48659a51b96661236 P 2c310c6a38754961a8795b0efbad0eab: Bootstrap starting.
I20260812 03:08:21.289064 32481 tablet_bootstrap.cc:492] T 6e37f44a332141e48659a51b96661236 P b11a569d8f3043709142c20a6c283287: No bootstrap required, opened a new log
I20260812 03:08:21.289309 32481 ts_tablet_manager.cc:1397] T 6e37f44a332141e48659a51b96661236 P b11a569d8f3043709142c20a6c283287: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:08:21.289665 32483 tablet_bootstrap.cc:654] T 6e37f44a332141e48659a51b96661236 P 2c310c6a38754961a8795b0efbad0eab: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:21.290897 32483 log.cc:824] T 6e37f44a332141e48659a51b96661236 P 2c310c6a38754961a8795b0efbad0eab: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:21.288362 32482 log.cc:824] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:21.293085 32483 tablet_bootstrap.cc:492] T 6e37f44a332141e48659a51b96661236 P 2c310c6a38754961a8795b0efbad0eab: No bootstrap required, opened a new log
I20260812 03:08:21.293321 32483 ts_tablet_manager.cc:1397] T 6e37f44a332141e48659a51b96661236 P 2c310c6a38754961a8795b0efbad0eab: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:08:21.295128 32481 raft_consensus.cc:348] T 6e37f44a332141e48659a51b96661236 P b11a569d8f3043709142c20a6c283287 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3cfe9fc204c4c9eff0d527ea7e980" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 42333 } } peers { permanent_uuid: "2c310c6a38754961a8795b0efbad0eab" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41645 } } peers { permanent_uuid: "b11a569d8f3043709142c20a6c283287" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 35715 } }
I20260812 03:08:21.295498 32481 raft_consensus.cc:374] T 6e37f44a332141e48659a51b96661236 P b11a569d8f3043709142c20a6c283287 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:21.295563 32481 raft_consensus.cc:729] T 6e37f44a332141e48659a51b96661236 P b11a569d8f3043709142c20a6c283287 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b11a569d8f3043709142c20a6c283287, State: Initialized, Role: FOLLOWER
I20260812 03:08:21.296109 32481 consensus_queue.cc:260] T 6e37f44a332141e48659a51b96661236 P b11a569d8f3043709142c20a6c283287 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3cfe9fc204c4c9eff0d527ea7e980" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 42333 } } peers { permanent_uuid: "2c310c6a38754961a8795b0efbad0eab" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41645 } } peers { permanent_uuid: "b11a569d8f3043709142c20a6c283287" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 35715 } }
I20260812 03:08:21.297385 32481 ts_tablet_manager.cc:1428] T 6e37f44a332141e48659a51b96661236 P b11a569d8f3043709142c20a6c283287: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
I20260812 03:08:21.297838 32184 heartbeater.cc:503] Master 127.26.250.190:36063 was elected leader, sending a full tablet report...
I20260812 03:08:21.297999 32482 tablet_bootstrap.cc:492] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980: No bootstrap required, opened a new log
I20260812 03:08:21.298580 32482 ts_tablet_manager.cc:1397] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260812 03:08:21.299049 32483 raft_consensus.cc:348] T 6e37f44a332141e48659a51b96661236 P 2c310c6a38754961a8795b0efbad0eab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3cfe9fc204c4c9eff0d527ea7e980" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 42333 } } peers { permanent_uuid: "2c310c6a38754961a8795b0efbad0eab" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41645 } } peers { permanent_uuid: "b11a569d8f3043709142c20a6c283287" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 35715 } }
I20260812 03:08:21.299365 32483 raft_consensus.cc:374] T 6e37f44a332141e48659a51b96661236 P 2c310c6a38754961a8795b0efbad0eab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:21.299445 32483 raft_consensus.cc:729] T 6e37f44a332141e48659a51b96661236 P 2c310c6a38754961a8795b0efbad0eab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c310c6a38754961a8795b0efbad0eab, State: Initialized, Role: FOLLOWER
I20260812 03:08:21.300005 32483 consensus_queue.cc:260] T 6e37f44a332141e48659a51b96661236 P 2c310c6a38754961a8795b0efbad0eab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3cfe9fc204c4c9eff0d527ea7e980" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 42333 } } peers { permanent_uuid: "2c310c6a38754961a8795b0efbad0eab" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41645 } } peers { permanent_uuid: "b11a569d8f3043709142c20a6c283287" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 35715 } }
I20260812 03:08:21.300809 32463 heartbeater.cc:503] Master 127.26.250.190:36063 was elected leader, sending a full tablet report...
I20260812 03:08:21.301172 32483 ts_tablet_manager.cc:1428] T 6e37f44a332141e48659a51b96661236 P 2c310c6a38754961a8795b0efbad0eab: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260812 03:08:21.302954 32482 raft_consensus.cc:348] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3cfe9fc204c4c9eff0d527ea7e980" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 42333 } } peers { permanent_uuid: "2c310c6a38754961a8795b0efbad0eab" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41645 } } peers { permanent_uuid: "b11a569d8f3043709142c20a6c283287" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 35715 } }
I20260812 03:08:21.303284 32482 raft_consensus.cc:374] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:21.303375 32482 raft_consensus.cc:729] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0f3cfe9fc204c4c9eff0d527ea7e980, State: Initialized, Role: FOLLOWER
I20260812 03:08:21.303925 32482 consensus_queue.cc:260] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3cfe9fc204c4c9eff0d527ea7e980" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 42333 } } peers { permanent_uuid: "2c310c6a38754961a8795b0efbad0eab" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41645 } } peers { permanent_uuid: "b11a569d8f3043709142c20a6c283287" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 35715 } }
I20260812 03:08:21.305171 32482 ts_tablet_manager.cc:1428] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 03:08:21.305333 32325 heartbeater.cc:503] Master 127.26.250.190:36063 was elected leader, sending a full tablet report...
I20260812 03:08:21.335649 32489 raft_consensus.cc:482] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:08:21.335987 32489 raft_consensus.cc:504] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3cfe9fc204c4c9eff0d527ea7e980" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 42333 } } peers { permanent_uuid: "2c310c6a38754961a8795b0efbad0eab" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41645 } } peers { permanent_uuid: "b11a569d8f3043709142c20a6c283287" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 35715 } }
I20260812 03:08:21.337467 32489 leader_election.cc:288] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c310c6a38754961a8795b0efbad0eab (127.26.250.131:41645), b11a569d8f3043709142c20a6c283287 (127.26.250.129:35715)
I20260812 03:08:21.341988 32478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:21.337979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:42563 (local address 127.26.250.131:41645)
0812 03:08:21.338133 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:21.338137 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:21.338155 (+ 18us) server_negotiation.cc:407] Connection header received
0812 03:08:21.338216 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:21.338219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:21.338279 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:21.338395 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:21.338754 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.339425 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.340776 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.341311 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.341419 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:21.341492 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:21.341745 (+ 253us) server_negotiation.cc:299] Negotiation successful
0812 03:08:21.341841 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 03:08:21.341992 32479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:21.337968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:51559 (local address 127.26.250.129:35715)
0812 03:08:21.338120 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:21.338127 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:21.338391 (+ 264us) server_negotiation.cc:407] Connection header received
0812 03:08:21.338519 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:21.338522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:21.338569 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:21.338658 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:21.338980 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.339593 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.340930 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.341395 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.341528 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:21.341579 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:21.341794 (+ 215us) server_negotiation.cc:299] Negotiation successful
0812 03:08:21.341870 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 03:08:21.341993 32327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:21.337713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:41645 (local address 127.26.250.130:42563)
0812 03:08:21.337831 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:21.337851 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:21.337964 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:21.338448 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:21.338454 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:21.338482 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:21.338624 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:21.338634 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.339553 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.339556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:21.340629 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:21.340639 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.341464 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.341470 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:21.341584 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:21.341612 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 03:08:21.341696 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 03:08:21.341803 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":3}
I20260812 03:08:21.342016 32490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:21.337837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:35715 (local address 127.26.250.130:51559)
0812 03:08:21.338249 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:21.338264 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:21.338412 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:21.338700 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:21.338705 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:21.338728 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:21.338856 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:21.338863 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.339711 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.339714 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:21.340786 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:21.340797 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.341540 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.341546 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:21.341640 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:21.341659 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 03:08:21.341767 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 03:08:21.341839 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":44,"thread_start_us":146,"threads_started":1}
I20260812 03:08:21.342804 32415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e37f44a332141e48659a51b96661236" candidate_uuid: "f0f3cfe9fc204c4c9eff0d527ea7e980" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c310c6a38754961a8795b0efbad0eab" is_pre_election: true
I20260812 03:08:21.342808 32134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e37f44a332141e48659a51b96661236" candidate_uuid: "f0f3cfe9fc204c4c9eff0d527ea7e980" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b11a569d8f3043709142c20a6c283287" is_pre_election: true
I20260812 03:08:21.343163 32134 raft_consensus.cc:2454] T 6e37f44a332141e48659a51b96661236 P b11a569d8f3043709142c20a6c283287 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0f3cfe9fc204c4c9eff0d527ea7e980 in term 0.
I20260812 03:08:21.343154 32415 raft_consensus.cc:2454] T 6e37f44a332141e48659a51b96661236 P 2c310c6a38754961a8795b0efbad0eab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0f3cfe9fc204c4c9eff0d527ea7e980 in term 0.
I20260812 03:08:21.343722 32209 leader_election.cc:302] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b11a569d8f3043709142c20a6c283287, f0f3cfe9fc204c4c9eff0d527ea7e980; no voters:
I20260812 03:08:21.344086 32489 raft_consensus.cc:2781] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:08:21.344213 32489 raft_consensus.cc:482] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:08:21.344274 32489 raft_consensus.cc:3037] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:21.345448 32489 raft_consensus.cc:504] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3cfe9fc204c4c9eff0d527ea7e980" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 42333 } } peers { permanent_uuid: "2c310c6a38754961a8795b0efbad0eab" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41645 } } peers { permanent_uuid: "b11a569d8f3043709142c20a6c283287" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 35715 } }
I20260812 03:08:21.345886 32489 leader_election.cc:288] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980 [CANDIDATE]: Term 1 election: Requested vote from peers 2c310c6a38754961a8795b0efbad0eab (127.26.250.131:41645), b11a569d8f3043709142c20a6c283287 (127.26.250.129:35715)
I20260812 03:08:21.346277 32415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e37f44a332141e48659a51b96661236" candidate_uuid: "f0f3cfe9fc204c4c9eff0d527ea7e980" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c310c6a38754961a8795b0efbad0eab"
I20260812 03:08:21.346403 32134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e37f44a332141e48659a51b96661236" candidate_uuid: "f0f3cfe9fc204c4c9eff0d527ea7e980" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b11a569d8f3043709142c20a6c283287"
I20260812 03:08:21.346536 32415 raft_consensus.cc:3037] T 6e37f44a332141e48659a51b96661236 P 2c310c6a38754961a8795b0efbad0eab [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:21.346565 32134 raft_consensus.cc:3037] T 6e37f44a332141e48659a51b96661236 P b11a569d8f3043709142c20a6c283287 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:21.347697 32134 raft_consensus.cc:2454] T 6e37f44a332141e48659a51b96661236 P b11a569d8f3043709142c20a6c283287 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0f3cfe9fc204c4c9eff0d527ea7e980 in term 1.
I20260812 03:08:21.347800 32415 raft_consensus.cc:2454] T 6e37f44a332141e48659a51b96661236 P 2c310c6a38754961a8795b0efbad0eab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0f3cfe9fc204c4c9eff0d527ea7e980 in term 1.
I20260812 03:08:21.348114 32209 leader_election.cc:302] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b11a569d8f3043709142c20a6c283287, f0f3cfe9fc204c4c9eff0d527ea7e980; no voters:
I20260812 03:08:21.348330 32489 raft_consensus.cc:2781] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:21.348618 32489 raft_consensus.cc:686] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980 [term 1 LEADER]: Becoming Leader. State: Replica: f0f3cfe9fc204c4c9eff0d527ea7e980, State: Running, Role: LEADER
I20260812 03:08:21.348982 32489 consensus_queue.cc:237] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3cfe9fc204c4c9eff0d527ea7e980" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 42333 } } peers { permanent_uuid: "2c310c6a38754961a8795b0efbad0eab" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41645 } } peers { permanent_uuid: "b11a569d8f3043709142c20a6c283287" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 35715 } }
I20260812 03:08:21.352710 31979 catalog_manager.cc:5495] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980 reported cstate change: term changed from 0 to 1, leader changed from <none> to f0f3cfe9fc204c4c9eff0d527ea7e980 (127.26.250.130). New cstate: current_term: 1 leader_uuid: "f0f3cfe9fc204c4c9eff0d527ea7e980" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0f3cfe9fc204c4c9eff0d527ea7e980" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 42333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c310c6a38754961a8795b0efbad0eab" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b11a569d8f3043709142c20a6c283287" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 35715 } health_report { overall_health: UNKNOWN } } }
I20260812 03:08:21.373652 32476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:21.369450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33146 (local address 127.26.250.130:42333)
0812 03:08:21.369619 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:21.369625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:21.369642 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:08:21.369880 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:21.369883 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:21.369944 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:21.370071 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:08:21.370376 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.370966 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.371944 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:21.372193 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:21.372318 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:21.372694 (+ 376us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:08:21.373167 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:08:21.373318 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:21.373391 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 03:08:21.373461 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 03:08:21.380244 32415 raft_consensus.cc:1261] T 6e37f44a332141e48659a51b96661236 P 2c310c6a38754961a8795b0efbad0eab [term 1 FOLLOWER]: Refusing update from remote peer f0f3cfe9fc204c4c9eff0d527ea7e980: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:21.380283 32134 raft_consensus.cc:1261] T 6e37f44a332141e48659a51b96661236 P b11a569d8f3043709142c20a6c283287 [term 1 FOLLOWER]: Refusing update from remote peer f0f3cfe9fc204c4c9eff0d527ea7e980: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:21.381089 32489 consensus_queue.cc:1035] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c310c6a38754961a8795b0efbad0eab" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:21.381336 32491 consensus_queue.cc:1035] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b11a569d8f3043709142c20a6c283287" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 35715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:21.390542 32496 mvcc.cc:205] Tried to move back new op lower bound from 7317520799245209600 to 7317520799128895488. Current Snapshot: MvccSnapshot[applied={T|T < 7317520799245209600}]
I20260812 03:08:21.399029 31979 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:21.399219 31979 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:21.402070 31979 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 89f216816a5b4254af638d6255c2ad25: Sending DeleteTablet for 3 replicas of tablet 6e37f44a332141e48659a51b96661236
I20260812 03:08:21.403086 32114 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e37f44a332141e48659a51b96661236 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:21 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:59016
I20260812 03:08:21.403136 32394 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e37f44a332141e48659a51b96661236 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:21 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:54484
I20260812 03:08:21.404006 32255 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e37f44a332141e48659a51b96661236 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:21 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:33144
I20260812 03:08:21.404264 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 32046
I20260812 03:08:21.410660 32507 tablet_replica.cc:330] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980: stopping tablet replica
I20260812 03:08:21.411085 32507 raft_consensus.cc:2229] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:08:21.411517 32507 raft_consensus.cc:2258] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:21.413401 32507 ts_tablet_manager.cc:1905] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:21.414915 32506 tablet_replica.cc:330] T 6e37f44a332141e48659a51b96661236 P 2c310c6a38754961a8795b0efbad0eab: stopping tablet replica
I20260812 03:08:21.415405 32506 raft_consensus.cc:2229] T 6e37f44a332141e48659a51b96661236 P 2c310c6a38754961a8795b0efbad0eab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:21.415818 32506 raft_consensus.cc:2258] T 6e37f44a332141e48659a51b96661236 P 2c310c6a38754961a8795b0efbad0eab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:21.416745 32507 ts_tablet_manager.cc:1918] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:08:21.416860 32507 log.cc:1196] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-1/wal/wals/6e37f44a332141e48659a51b96661236
I20260812 03:08:21.417210 32507 ts_tablet_manager.cc:1939] T 6e37f44a332141e48659a51b96661236 P f0f3cfe9fc204c4c9eff0d527ea7e980: Deleting consensus metadata
W20260812 03:08:21.417904 31965 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.250.129:35715 (error 108)
I20260812 03:08:21.418663 31963 catalog_manager.cc:4841] TS f0f3cfe9fc204c4c9eff0d527ea7e980 (127.26.250.130:42333): tablet 6e37f44a332141e48659a51b96661236 (table test-table [id=f66dbc8b8a4d4a269f16868b445c9b23]) successfully deleted
W20260812 03:08:21.419329 32477 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:21.418791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:35715 (local address 127.0.0.1:59020)
0812 03:08:21.418927 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:21.418944 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:21.419077 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:21.419196 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.129:35715: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 03:08:21.419461 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 32190
W20260812 03:08:21.419529 31965 catalog_manager.cc:4584] TS b11a569d8f3043709142c20a6c283287 (127.26.250.129:35715): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6e37f44a332141e48659a51b96661236: Network error: Client connection negotiation failed: client connection to 127.26.250.129:35715: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260812 03:08:21.422102 32506 ts_tablet_manager.cc:1905] T 6e37f44a332141e48659a51b96661236 P 2c310c6a38754961a8795b0efbad0eab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:21.424777 32506 ts_tablet_manager.cc:1918] T 6e37f44a332141e48659a51b96661236 P 2c310c6a38754961a8795b0efbad0eab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:08:21.424896 32506 log.cc:1196] T 6e37f44a332141e48659a51b96661236 P 2c310c6a38754961a8795b0efbad0eab: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/6e37f44a332141e48659a51b96661236
I20260812 03:08:21.425244 32506 ts_tablet_manager.cc:1939] T 6e37f44a332141e48659a51b96661236 P 2c310c6a38754961a8795b0efbad0eab: Deleting consensus metadata
I20260812 03:08:21.426687 31962 catalog_manager.cc:4841] TS 2c310c6a38754961a8795b0efbad0eab (127.26.250.131:41645): tablet 6e37f44a332141e48659a51b96661236 (table test-table [id=f66dbc8b8a4d4a269f16868b445c9b23]) successfully deleted
I20260812 03:08:21.429136 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 32330
I20260812 03:08:21.437115 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 31946
2026-08-12T03:08:21Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2491 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T03:08:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:08:21Z Disabled control of system clock
I20260812 03:08:21.473673 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:36783
--webserver_interface=127.26.250.190
--webserver_port=0
--builtin_ntp_servers=127.26.250.148:46231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:36783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 03:08:21.603237 32512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:21.603565 32512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:21.603608 32512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:21.608270 32512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:08:21.608348 32512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:21.608366 32512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:21.608386 32512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:08:21.608404 32512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:08:21.613921 32512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:46231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:36783
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:36783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:21.615226 32512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:21.616760 32512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:21.626093 32518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:21.626147 32520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:21.626142 32517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:21.626197 32512 server_base.cc:1034] running on GCE node
I20260812 03:08:21.627143 32512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:21.628367 32512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:21.629580 32512 hybrid_clock.cc:648] HybridClock initialized: now 1786504101629542 us; error 55 us; skew 500 ppm
I20260812 03:08:21.631945 32512 webserver.cc:460] Webserver started at http://127.26.250.190:41441/ using document root <none> and password file <none>
I20260812 03:08:21.632609 32512 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:21.632715 32512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:21.633210 32512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:21.635635 32512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "b776008ef5ad49e2a630ecde7c6aae79"
format_stamp: "Formatted at 2026-08-12 03:08:21 on dist-test-slave-zpfg"
server_key: "ab7a354c476779d0cee49fc303808879"
server_key_iv: "a5eb6834369aa2cbb629d440cfbfb21e"
server_key_version: "encryptionkey@0"
I20260812 03:08:21.636246 32512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "b776008ef5ad49e2a630ecde7c6aae79"
format_stamp: "Formatted at 2026-08-12 03:08:21 on dist-test-slave-zpfg"
server_key: "ab7a354c476779d0cee49fc303808879"
server_key_iv: "a5eb6834369aa2cbb629d440cfbfb21e"
server_key_version: "encryptionkey@0"
I20260812 03:08:21.640264 32512 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:08:21.643028 32526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:21.644447 32512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:08:21.644599 32512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "b776008ef5ad49e2a630ecde7c6aae79"
format_stamp: "Formatted at 2026-08-12 03:08:21 on dist-test-slave-zpfg"
server_key: "ab7a354c476779d0cee49fc303808879"
server_key_iv: "a5eb6834369aa2cbb629d440cfbfb21e"
server_key_version: "encryptionkey@0"
I20260812 03:08:21.644724 32512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:21.649801 32512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:21.650619 32512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:21.650805 32512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:21.658177 32512 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:36783
I20260812 03:08:21.658201 32581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:36783 every 8 connection(s)
I20260812 03:08:21.659211 32512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:08:21.661000 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 32512
I20260812 03:08:21.661103 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:08:21.661432 27626 external_mini_cluster.cc:1428] Setting key 81501f666d4d53fae4ceb5e929aaa253
I20260812 03:08:21.663009 32582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:21.667836 32585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:21.662981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43564 (local address 127.26.250.190:36783)
0812 03:08:21.663758 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:21.663777 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:21.663818 (+ 41us) server_negotiation.cc:407] Connection header received
0812 03:08:21.664581 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:21.664600 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:21.664986 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:21.665312 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:21.666220 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:21.666296 (+ 76us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:21.666309 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:08:21.666320 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:21.666445 (+ 125us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:08:21.666467 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:21.666591 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:21.666754 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 03:08:21.667037 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":176,"threads_started":1}
I20260812 03:08:21.670140 32582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b776008ef5ad49e2a630ecde7c6aae79: Bootstrap starting.
I20260812 03:08:21.672498 32582 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b776008ef5ad49e2a630ecde7c6aae79: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:21.673285 32582 log.cc:824] T 00000000000000000000000000000000 P b776008ef5ad49e2a630ecde7c6aae79: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:21.675730 32582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b776008ef5ad49e2a630ecde7c6aae79: No bootstrap required, opened a new log
I20260812 03:08:21.679890 32582 raft_consensus.cc:348] T 00000000000000000000000000000000 P b776008ef5ad49e2a630ecde7c6aae79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b776008ef5ad49e2a630ecde7c6aae79" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 36783 } }
I20260812 03:08:21.680140 32582 raft_consensus.cc:374] T 00000000000000000000000000000000 P b776008ef5ad49e2a630ecde7c6aae79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:21.680177 32582 raft_consensus.cc:729] T 00000000000000000000000000000000 P b776008ef5ad49e2a630ecde7c6aae79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b776008ef5ad49e2a630ecde7c6aae79, State: Initialized, Role: FOLLOWER
I20260812 03:08:21.680667 32582 consensus_queue.cc:260] T 00000000000000000000000000000000 P b776008ef5ad49e2a630ecde7c6aae79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b776008ef5ad49e2a630ecde7c6aae79" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 36783 } }
I20260812 03:08:21.680851 32582 raft_consensus.cc:388] T 00000000000000000000000000000000 P b776008ef5ad49e2a630ecde7c6aae79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:08:21.680925 32582 raft_consensus.cc:482] T 00000000000000000000000000000000 P b776008ef5ad49e2a630ecde7c6aae79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:08:21.681048 32582 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b776008ef5ad49e2a630ecde7c6aae79 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:21.682024 32582 raft_consensus.cc:504] T 00000000000000000000000000000000 P b776008ef5ad49e2a630ecde7c6aae79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b776008ef5ad49e2a630ecde7c6aae79" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 36783 } }
I20260812 03:08:21.682440 32582 leader_election.cc:302] T 00000000000000000000000000000000 P b776008ef5ad49e2a630ecde7c6aae79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b776008ef5ad49e2a630ecde7c6aae79; no voters:
I20260812 03:08:21.682792 32582 leader_election.cc:288] T 00000000000000000000000000000000 P b776008ef5ad49e2a630ecde7c6aae79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:08:21.682941 32587 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b776008ef5ad49e2a630ecde7c6aae79 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:21.683218 32587 raft_consensus.cc:686] T 00000000000000000000000000000000 P b776008ef5ad49e2a630ecde7c6aae79 [term 1 LEADER]: Becoming Leader. State: Replica: b776008ef5ad49e2a630ecde7c6aae79, State: Running, Role: LEADER
I20260812 03:08:21.683537 32587 consensus_queue.cc:237] T 00000000000000000000000000000000 P b776008ef5ad49e2a630ecde7c6aae79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b776008ef5ad49e2a630ecde7c6aae79" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 36783 } }
I20260812 03:08:21.684149 32582 sys_catalog.cc:564] T 00000000000000000000000000000000 P b776008ef5ad49e2a630ecde7c6aae79 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:08:21.685283 32589 sys_catalog.cc:455] T 00000000000000000000000000000000 P b776008ef5ad49e2a630ecde7c6aae79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b776008ef5ad49e2a630ecde7c6aae79. Latest consensus state: current_term: 1 leader_uuid: "b776008ef5ad49e2a630ecde7c6aae79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b776008ef5ad49e2a630ecde7c6aae79" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 36783 } } }
I20260812 03:08:21.685412 32589 sys_catalog.cc:458] T 00000000000000000000000000000000 P b776008ef5ad49e2a630ecde7c6aae79 [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:21.685617 32588 sys_catalog.cc:455] T 00000000000000000000000000000000 P b776008ef5ad49e2a630ecde7c6aae79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b776008ef5ad49e2a630ecde7c6aae79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b776008ef5ad49e2a630ecde7c6aae79" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 36783 } } }
I20260812 03:08:21.685771 32588 sys_catalog.cc:458] T 00000000000000000000000000000000 P b776008ef5ad49e2a630ecde7c6aae79 [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:21.685858 32596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:08:21.688920 32596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:08:21.694859 32596 catalog_manager.cc:1347] Generated new cluster ID: 7068abcb88044d79a7973ddb924be62d
I20260812 03:08:21.694954 32596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:08:21.735015 32596 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:08:21.735424 32596 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:08:21.744443 32596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b776008ef5ad49e2a630ecde7c6aae79: Generated new TSK 0
I20260812 03:08:21.745370 32596 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:08:21.759248 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:0
--local_ip_for_outbound_sockets=127.26.250.129
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:36783
--builtin_ntp_servers=127.26.250.148:46231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 03:08:21.881935 32608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:21.882246 32608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:21.882285 32608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:21.886839 32608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:21.886909 32608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:21.886968 32608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:08:21.891968 32608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:46231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:36783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:21.893265 32608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:21.894860 32608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:21.906039 32613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:21.905984 32616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:21.906001 32614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:21.906059 32608 server_base.cc:1034] running on GCE node
I20260812 03:08:21.906821 32608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:21.907842 32608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:21.909090 32608 hybrid_clock.cc:648] HybridClock initialized: now 1786504101909072 us; error 31 us; skew 500 ppm
I20260812 03:08:21.911682 32608 webserver.cc:460] Webserver started at http://127.26.250.129:41693/ using document root <none> and password file <none>
I20260812 03:08:21.912410 32608 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:21.912525 32608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:21.912807 32608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:21.915302 32608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-0/data/instance:
uuid: "fb5666afb66f4232ba329f9c7d63f269"
format_stamp: "Formatted at 2026-08-12 03:08:21 on dist-test-slave-zpfg"
server_key: "3b8ef4373f03c3f0e83f13547654ca4f"
server_key_iv: "51dabf9010d18fb31c6d42538ea2eb25"
server_key_version: "encryptionkey@0"
I20260812 03:08:21.915971 32608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance:
uuid: "fb5666afb66f4232ba329f9c7d63f269"
format_stamp: "Formatted at 2026-08-12 03:08:21 on dist-test-slave-zpfg"
server_key: "3b8ef4373f03c3f0e83f13547654ca4f"
server_key_iv: "51dabf9010d18fb31c6d42538ea2eb25"
server_key_version: "encryptionkey@0"
I20260812 03:08:21.920382 32608 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:08:21.923592 32622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:21.925073 32608 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:08:21.925247 32608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "fb5666afb66f4232ba329f9c7d63f269"
format_stamp: "Formatted at 2026-08-12 03:08:21 on dist-test-slave-zpfg"
server_key: "3b8ef4373f03c3f0e83f13547654ca4f"
server_key_iv: "51dabf9010d18fb31c6d42538ea2eb25"
server_key_version: "encryptionkey@0"
I20260812 03:08:21.925398 32608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:21.930444 32608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:21.931218 32608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:21.931413 32608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:21.931996 32608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:21.932919 32608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:21.932965 32608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:21.933004 32608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:21.933028 32608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:21.942651 32608 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:34675
I20260812 03:08:21.942675 32743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:34675 every 8 connection(s)
I20260812 03:08:21.943704 32608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:08:21.947120 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 32608
I20260812 03:08:21.947258 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance
I20260812 03:08:21.947587 27626 external_mini_cluster.cc:1428] Setting key 11a4de1d1529e9dac215397e5c7ee065
I20260812 03:08:21.949887 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:0
--local_ip_for_outbound_sockets=127.26.250.130
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:36783
--builtin_ntp_servers=127.26.250.148:46231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 03:08:21.950943 32585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:21.945660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:52631 (local address 127.26.250.190:36783)
0812 03:08:21.945824 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:21.945828 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:21.947151 (+ 1323us) server_negotiation.cc:407] Connection header received
0812 03:08:21.948989 (+ 1838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:21.948994 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:21.949039 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:21.949114 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:21.950485 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:21.950488 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:21.950490 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:08:21.950501 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:21.950522 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:08:21.950530 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:21.950608 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:21.950790 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 03:08:21.950829 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 03:08:21.952185 32746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:21.946099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:36783 (local address 127.26.250.129:52631)
0812 03:08:21.946946 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:21.947011 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:21.948773 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:21.949315 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:21.949325 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:21.949457 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:21.950126 (+ 669us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:08:21.950130 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:21.950190 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:08:21.950286 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:21.950624 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:21.950636 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:21.950645 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 03:08:21.950850 (+ 205us) client_negotiation.cc:240] Negotiation successful
0812 03:08:21.951121 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":122,"threads_started":1}
I20260812 03:08:21.953579 32744 heartbeater.cc:347] Connected to a master server at 127.26.250.190:36783
I20260812 03:08:21.953847 32744 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:21.954286 32744 heartbeater.cc:511] Master 127.26.250.190:36783 requested a full tablet report, sending...
I20260812 03:08:21.956066 32544 ts_manager.cc:194] Registered new tserver with Master: fb5666afb66f4232ba329f9c7d63f269 (127.26.250.129:34675)
W20260812 03:08:22.076711 32747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:22.077064 32747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:22.077159 32747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:22.081832 32747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:22.081939 32747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:22.082049 32747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:08:22.087855 32747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:46231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:36783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:22.089498 32747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:22.091549 32747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:22.102073 32753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:22.102073 32755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:22.102087 32752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:22.102319 32747 server_base.cc:1034] running on GCE node
I20260812 03:08:22.102982 32747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:22.103663 32747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:22.104869 32747 hybrid_clock.cc:648] HybridClock initialized: now 1786504102104855 us; error 30 us; skew 500 ppm
I20260812 03:08:22.107041 32747 webserver.cc:460] Webserver started at http://127.26.250.130:46575/ using document root <none> and password file <none>
I20260812 03:08:22.107690 32747 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:22.107795 32747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:22.108040 32747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:22.110194 32747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-1/data/instance:
uuid: "0d03837ac8c440a58057b18c0312e60f"
format_stamp: "Formatted at 2026-08-12 03:08:22 on dist-test-slave-zpfg"
server_key: "0033d5de542aca2d7fbdaa90fe2c771b"
server_key_iv: "90b16ef6c3d4b73010c24e44ca75f421"
server_key_version: "encryptionkey@0"
I20260812 03:08:22.110880 32747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance:
uuid: "0d03837ac8c440a58057b18c0312e60f"
format_stamp: "Formatted at 2026-08-12 03:08:22 on dist-test-slave-zpfg"
server_key: "0033d5de542aca2d7fbdaa90fe2c771b"
server_key_iv: "90b16ef6c3d4b73010c24e44ca75f421"
server_key_version: "encryptionkey@0"
I20260812 03:08:22.114910 32747 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:08:22.117820 32761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:22.119205 32747 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:08:22.119356 32747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "0d03837ac8c440a58057b18c0312e60f"
format_stamp: "Formatted at 2026-08-12 03:08:22 on dist-test-slave-zpfg"
server_key: "0033d5de542aca2d7fbdaa90fe2c771b"
server_key_iv: "90b16ef6c3d4b73010c24e44ca75f421"
server_key_version: "encryptionkey@0"
I20260812 03:08:22.119489 32747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:22.125018 32747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:22.125792 32747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:22.126011 32747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:22.126731 32747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:22.127801 32747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:22.127893 32747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:22.127974 32747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:22.128032 32747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:22.138824 32747 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:45749
I20260812 03:08:22.138959 416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:45749 every 8 connection(s)
I20260812 03:08:22.140246 32747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:08:22.148097 32585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:22.142695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:52013 (local address 127.26.250.190:36783)
0812 03:08:22.142850 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:22.142854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:22.144188 (+ 1334us) server_negotiation.cc:407] Connection header received
0812 03:08:22.146168 (+ 1980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:22.146171 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:22.146216 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:22.146298 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:22.147609 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:22.147613 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:22.147615 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:08:22.147628 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:22.147650 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:08:22.147659 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:22.147748 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:22.147900 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 03:08:22.147960 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:08:22.148612 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 32747
I20260812 03:08:22.148741 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance
I20260812 03:08:22.149076 27626 external_mini_cluster.cc:1428] Setting key 2a19fff47e00e007559780bad4065d31
I20260812 03:08:22.149712 419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:22.143212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:36783 (local address 127.26.250.130:52013)
0812 03:08:22.143985 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:22.144045 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:22.145893 (+ 1848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:22.146592 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:22.146601 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:22.146709 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:22.147306 (+ 597us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:08:22.147308 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:22.147361 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:08:22.147441 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:22.147745 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:22.147753 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:22.147757 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:08:22.148132 (+ 375us) client_negotiation.cc:240] Negotiation successful
0812 03:08:22.148570 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":133,"threads_started":1}
I20260812 03:08:22.151307 417 heartbeater.cc:347] Connected to a master server at 127.26.250.190:36783
I20260812 03:08:22.151592 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:0
--local_ip_for_outbound_sockets=127.26.250.131
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:36783
--builtin_ntp_servers=127.26.250.148:46231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 03:08:22.151604 417 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:22.152122 417 heartbeater.cc:511] Master 127.26.250.190:36783 requested a full tablet report, sending...
I20260812 03:08:22.153409 32543 ts_manager.cc:194] Registered new tserver with Master: 0d03837ac8c440a58057b18c0312e60f (127.26.250.130:45749)
W20260812 03:08:22.276258 420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:22.276638 420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:22.276718 420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:22.281041 420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:22.281168 420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:22.281282 420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:08:22.286821 420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:46231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:36783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:22.288106 420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:22.289789 420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:22.300993 426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:22.301015 428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:22.300993 425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:22.301339 420 server_base.cc:1034] running on GCE node
I20260812 03:08:22.301878 420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:22.302804 420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:22.304015 420 hybrid_clock.cc:648] HybridClock initialized: now 1786504102303966 us; error 63 us; skew 500 ppm
I20260812 03:08:22.306598 420 webserver.cc:460] Webserver started at http://127.26.250.131:34549/ using document root <none> and password file <none>
I20260812 03:08:22.307336 420 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:22.307406 420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:22.307976 420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:22.310460 420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-2/data/instance:
uuid: "f595c4d2058a4ef087b21d9156cc1c3d"
format_stamp: "Formatted at 2026-08-12 03:08:22 on dist-test-slave-zpfg"
server_key: "0747be89e6147464fbe53f52a9ddcc32"
server_key_iv: "a90e4376f69b7f793a4a8c9e857f2171"
server_key_version: "encryptionkey@0"
I20260812 03:08:22.311120 420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance:
uuid: "f595c4d2058a4ef087b21d9156cc1c3d"
format_stamp: "Formatted at 2026-08-12 03:08:22 on dist-test-slave-zpfg"
server_key: "0747be89e6147464fbe53f52a9ddcc32"
server_key_iv: "a90e4376f69b7f793a4a8c9e857f2171"
server_key_version: "encryptionkey@0"
I20260812 03:08:22.315424 420 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:08:22.318481 435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:22.319883 420 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 03:08:22.320044 420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "f595c4d2058a4ef087b21d9156cc1c3d"
format_stamp: "Formatted at 2026-08-12 03:08:22 on dist-test-slave-zpfg"
server_key: "0747be89e6147464fbe53f52a9ddcc32"
server_key_iv: "a90e4376f69b7f793a4a8c9e857f2171"
server_key_version: "encryptionkey@0"
I20260812 03:08:22.320186 420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:22.325587 420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:22.326293 420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:22.326546 420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:22.327145 420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:22.328130 420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:22.328207 420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:22.328279 420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:22.328313 420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:22.338478 420 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:33851
I20260812 03:08:22.338507 554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:33851 every 8 connection(s)
I20260812 03:08:22.339475 420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:08:22.340322 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 420
I20260812 03:08:22.340456 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance
I20260812 03:08:22.340725 27626 external_mini_cluster.cc:1428] Setting key 2d6d94a3cc3e5e4ed1cf157883f7e618
I20260812 03:08:22.346910 32585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:22.342298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:41341 (local address 127.26.250.190:36783)
0812 03:08:22.342488 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:22.342492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:22.343622 (+ 1130us) server_negotiation.cc:407] Connection header received
0812 03:08:22.345043 (+ 1421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:22.345047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:22.345092 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:22.345169 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:22.346403 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:22.346406 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:22.346409 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:08:22.346422 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:22.346451 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:08:22.346459 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:22.346563 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:22.346715 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 03:08:22.346766 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260812 03:08:22.347875 557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:22.342760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:36783 (local address 127.26.250.131:41341)
0812 03:08:22.343472 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:22.343514 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:22.344774 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:22.345360 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:22.345372 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:22.345480 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:22.346098 (+ 618us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:08:22.346100 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:22.346142 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:08:22.346213 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:22.346554 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:22.346564 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:22.346572 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 03:08:22.346770 (+ 198us) client_negotiation.cc:240] Negotiation successful
0812 03:08:22.347081 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":126,"threads_started":1}
I20260812 03:08:22.349061 555 heartbeater.cc:347] Connected to a master server at 127.26.250.190:36783
I20260812 03:08:22.349431 555 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:22.349985 555 heartbeater.cc:511] Master 127.26.250.190:36783 requested a full tablet report, sending...
I20260812 03:08:22.351114 32543 ts_manager.cc:194] Registered new tserver with Master: f595c4d2058a4ef087b21d9156cc1c3d (127.26.250.131:33851)
I20260812 03:08:22.354918 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:08:22.359494 32585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:22.357131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43580 (local address 127.26.250.190:36783)
0812 03:08:22.357426 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:22.357431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:22.357585 (+ 154us) server_negotiation.cc:407] Connection header received
0812 03:08:22.357766 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:22.357772 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:22.357811 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:22.357909 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:22.358894 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:22.358898 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:22.358901 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:08:22.358913 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:22.358946 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:08:22.358955 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:22.359056 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:22.359319 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 03:08:22.359365 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181}
I20260812 03:08:22.362865 32543 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43580:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:08:22.366029 32543 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:08:22.379724 567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:22.375532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:33851 (local address 127.0.0.1:41028)
0812 03:08:22.376017 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:22.376081 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:22.376210 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:22.377664 (+ 1454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:22.377667 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:22.377727 (+ 60us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:22.378906 (+ 1179us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:08:22.378910 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:22.378920 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:08:22.378946 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:22.379420 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:22.379422 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:22.379425 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 03:08:22.379484 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 03:08:22.379564 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":111,"threads_started":1}
I20260812 03:08:22.379845 570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:22.376539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37920 (local address 127.26.250.130:45749)
0812 03:08:22.377103 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:22.377116 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:22.377266 (+ 150us) server_negotiation.cc:407] Connection header received
0812 03:08:22.377323 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:22.377327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:22.377455 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:22.377588 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:22.379092 (+ 1504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:22.379098 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:22.379111 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:08:22.379124 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:22.379173 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:08:22.379209 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:22.379316 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:22.379600 (+ 284us) server_negotiation.cc:299] Negotiation successful
0812 03:08:22.379678 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":150,"threads_started":1}
I20260812 03:08:22.380015 571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:22.376669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57392 (local address 127.26.250.129:34675)
0812 03:08:22.377103 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:22.377115 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:22.377139 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:08:22.377196 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:22.377200 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:22.377433 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:22.377588 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:22.378949 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:22.379159 (+ 210us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:22.379172 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:08:22.379188 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:22.379247 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:08:22.379289 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:22.379669 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:22.379734 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 03:08:22.379809 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":108,"threads_started":1}
I20260812 03:08:22.380246 566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:22.375573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:34675 (local address 127.0.0.1:57392)
0812 03:08:22.376721 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:22.376738 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:22.376822 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:22.377636 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:22.377644 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:22.377748 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:22.378600 (+ 852us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:08:22.378604 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:22.378631 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:08:22.378772 (+ 141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:22.379399 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:22.379402 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:22.379406 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:08:22.379479 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 03:08:22.379563 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1083,"mutex_wait_us":3,"spinlock_wait_cycles":2432,"thread_start_us":106,"threads_started":1}
I20260812 03:08:22.380581 569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:22.376371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:45749 (local address 127.0.0.1:37920)
0812 03:08:22.377076 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:22.377090 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:22.377248 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:22.378319 (+ 1071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:22.378321 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:22.378382 (+ 61us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:22.378894 (+ 512us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:08:22.378898 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:22.378910 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:08:22.378939 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:22.379366 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:22.379377 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:22.379385 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 03:08:22.379491 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 03:08:22.380503 (+ 1012us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"mutex_wait_us":946,"thread_start_us":97,"threads_started":1}
I20260812 03:08:22.381079 568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:22.375721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41028 (local address 127.26.250.131:33851)
0812 03:08:22.377059 (+ 1338us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:22.377072 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:22.377101 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:08:22.377161 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:22.377166 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:22.377456 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:22.378471 (+ 1015us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:22.379212 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:22.379217 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:22.379227 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:08:22.379241 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:22.379277 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:08:22.379301 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:22.380813 (+ 1512us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:22.380865 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 03:08:22.380938 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1217,"thread_start_us":141,"threads_started":1}
I20260812 03:08:22.383368 32673 tablet_service.cc:1467] Processing CreateTablet for tablet 077439a4cb1d4b2b9a6ccd807e9d23c7 (DEFAULT_TABLE table=test-table [id=e0dd368ecc97445184a425831b0724df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:22.383790 485 tablet_service.cc:1467] Processing CreateTablet for tablet 077439a4cb1d4b2b9a6ccd807e9d23c7 (DEFAULT_TABLE table=test-table [id=e0dd368ecc97445184a425831b0724df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:22.383667 343 tablet_service.cc:1467] Processing CreateTablet for tablet 077439a4cb1d4b2b9a6ccd807e9d23c7 (DEFAULT_TABLE table=test-table [id=e0dd368ecc97445184a425831b0724df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:22.384581 32673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 077439a4cb1d4b2b9a6ccd807e9d23c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:22.384852 343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 077439a4cb1d4b2b9a6ccd807e9d23c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:22.384852 485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 077439a4cb1d4b2b9a6ccd807e9d23c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:22.391503 572 tablet_bootstrap.cc:492] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P fb5666afb66f4232ba329f9c7d63f269: Bootstrap starting.
I20260812 03:08:22.392851 573 tablet_bootstrap.cc:492] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P 0d03837ac8c440a58057b18c0312e60f: Bootstrap starting.
I20260812 03:08:22.394735 574 tablet_bootstrap.cc:492] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P f595c4d2058a4ef087b21d9156cc1c3d: Bootstrap starting.
I20260812 03:08:22.394825 572 tablet_bootstrap.cc:654] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P fb5666afb66f4232ba329f9c7d63f269: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:22.394881 573 tablet_bootstrap.cc:654] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P 0d03837ac8c440a58057b18c0312e60f: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:22.395637 572 log.cc:824] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P fb5666afb66f4232ba329f9c7d63f269: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:22.395733 573 log.cc:824] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P 0d03837ac8c440a58057b18c0312e60f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:22.397251 574 tablet_bootstrap.cc:654] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P f595c4d2058a4ef087b21d9156cc1c3d: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:22.398073 572 tablet_bootstrap.cc:492] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P fb5666afb66f4232ba329f9c7d63f269: No bootstrap required, opened a new log
I20260812 03:08:22.398144 574 log.cc:824] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P f595c4d2058a4ef087b21d9156cc1c3d: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:22.398294 572 ts_tablet_manager.cc:1397] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P fb5666afb66f4232ba329f9c7d63f269: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 03:08:22.399179 573 tablet_bootstrap.cc:492] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P 0d03837ac8c440a58057b18c0312e60f: No bootstrap required, opened a new log
I20260812 03:08:22.399477 573 ts_tablet_manager.cc:1397] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P 0d03837ac8c440a58057b18c0312e60f: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 03:08:22.400192 574 tablet_bootstrap.cc:492] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P f595c4d2058a4ef087b21d9156cc1c3d: No bootstrap required, opened a new log
I20260812 03:08:22.400385 574 ts_tablet_manager.cc:1397] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P f595c4d2058a4ef087b21d9156cc1c3d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 03:08:22.403198 572 raft_consensus.cc:348] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P fb5666afb66f4232ba329f9c7d63f269 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f595c4d2058a4ef087b21d9156cc1c3d" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 33851 } } peers { permanent_uuid: "fb5666afb66f4232ba329f9c7d63f269" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34675 } } peers { permanent_uuid: "0d03837ac8c440a58057b18c0312e60f" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45749 } }
I20260812 03:08:22.403558 572 raft_consensus.cc:374] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P fb5666afb66f4232ba329f9c7d63f269 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:22.403653 572 raft_consensus.cc:729] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P fb5666afb66f4232ba329f9c7d63f269 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb5666afb66f4232ba329f9c7d63f269, State: Initialized, Role: FOLLOWER
I20260812 03:08:22.403456 573 raft_consensus.cc:348] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P 0d03837ac8c440a58057b18c0312e60f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f595c4d2058a4ef087b21d9156cc1c3d" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 33851 } } peers { permanent_uuid: "fb5666afb66f4232ba329f9c7d63f269" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34675 } } peers { permanent_uuid: "0d03837ac8c440a58057b18c0312e60f" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45749 } }
I20260812 03:08:22.403764 573 raft_consensus.cc:374] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P 0d03837ac8c440a58057b18c0312e60f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:22.403841 573 raft_consensus.cc:729] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P 0d03837ac8c440a58057b18c0312e60f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d03837ac8c440a58057b18c0312e60f, State: Initialized, Role: FOLLOWER
I20260812 03:08:22.404178 572 consensus_queue.cc:260] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P fb5666afb66f4232ba329f9c7d63f269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f595c4d2058a4ef087b21d9156cc1c3d" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 33851 } } peers { permanent_uuid: "fb5666afb66f4232ba329f9c7d63f269" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34675 } } peers { permanent_uuid: "0d03837ac8c440a58057b18c0312e60f" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45749 } }
I20260812 03:08:22.404372 573 consensus_queue.cc:260] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P 0d03837ac8c440a58057b18c0312e60f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f595c4d2058a4ef087b21d9156cc1c3d" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 33851 } } peers { permanent_uuid: "fb5666afb66f4232ba329f9c7d63f269" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34675 } } peers { permanent_uuid: "0d03837ac8c440a58057b18c0312e60f" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45749 } }
I20260812 03:08:22.405071 574 raft_consensus.cc:348] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P f595c4d2058a4ef087b21d9156cc1c3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f595c4d2058a4ef087b21d9156cc1c3d" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 33851 } } peers { permanent_uuid: "fb5666afb66f4232ba329f9c7d63f269" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34675 } } peers { permanent_uuid: "0d03837ac8c440a58057b18c0312e60f" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45749 } }
I20260812 03:08:22.405355 572 ts_tablet_manager.cc:1428] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P fb5666afb66f4232ba329f9c7d63f269: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 03:08:22.405354 574 raft_consensus.cc:374] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P f595c4d2058a4ef087b21d9156cc1c3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:22.405431 574 raft_consensus.cc:729] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P f595c4d2058a4ef087b21d9156cc1c3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f595c4d2058a4ef087b21d9156cc1c3d, State: Initialized, Role: FOLLOWER
I20260812 03:08:22.405542 573 ts_tablet_manager.cc:1428] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P 0d03837ac8c440a58057b18c0312e60f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 03:08:22.405802 32744 heartbeater.cc:503] Master 127.26.250.190:36783 was elected leader, sending a full tablet report...
I20260812 03:08:22.405906 580 raft_consensus.cc:482] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P fb5666afb66f4232ba329f9c7d63f269 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:08:22.405885 574 consensus_queue.cc:260] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P f595c4d2058a4ef087b21d9156cc1c3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f595c4d2058a4ef087b21d9156cc1c3d" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 33851 } } peers { permanent_uuid: "fb5666afb66f4232ba329f9c7d63f269" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34675 } } peers { permanent_uuid: "0d03837ac8c440a58057b18c0312e60f" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45749 } }
I20260812 03:08:22.406030 580 raft_consensus.cc:504] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P fb5666afb66f4232ba329f9c7d63f269 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f595c4d2058a4ef087b21d9156cc1c3d" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 33851 } } peers { permanent_uuid: "fb5666afb66f4232ba329f9c7d63f269" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34675 } } peers { permanent_uuid: "0d03837ac8c440a58057b18c0312e60f" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45749 } }
I20260812 03:08:22.406385 417 heartbeater.cc:503] Master 127.26.250.190:36783 was elected leader, sending a full tablet report...
I20260812 03:08:22.406916 574 ts_tablet_manager.cc:1428] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P f595c4d2058a4ef087b21d9156cc1c3d: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 03:08:22.407258 555 heartbeater.cc:503] Master 127.26.250.190:36783 was elected leader, sending a full tablet report...
I20260812 03:08:22.408656 580 leader_election.cc:288] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P fb5666afb66f4232ba329f9c7d63f269 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f595c4d2058a4ef087b21d9156cc1c3d (127.26.250.131:33851), 0d03837ac8c440a58057b18c0312e60f (127.26.250.130:45749)
I20260812 03:08:22.410552 32746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:22.407821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:33851 (local address 127.26.250.129:58567)
0812 03:08:22.408126 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:22.408145 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:22.408242 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:22.409095 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:22.409101 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:22.409121 (+ 20us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:22.409762 (+ 641us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:08:22.409765 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:22.409787 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:08:22.409847 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:22.410250 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:22.410253 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:22.410257 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:08:22.410383 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 03:08:22.410427 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215}
I20260812 03:08:22.410948 568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:22.408141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:58567 (local address 127.26.250.131:33851)
0812 03:08:22.408264 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:22.408268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:22.408289 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:08:22.408330 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:22.408333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:22.408368 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:22.408451 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:22.410019 (+ 1568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:22.410026 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:22.410031 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:08:22.410050 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:22.410079 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:08:22.410093 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:22.410199 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:22.410755 (+ 556us) server_negotiation.cc:299] Negotiation successful
0812 03:08:22.410804 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:08:22.411772 505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077439a4cb1d4b2b9a6ccd807e9d23c7" candidate_uuid: "fb5666afb66f4232ba329f9c7d63f269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f595c4d2058a4ef087b21d9156cc1c3d" is_pre_election: true
I20260812 03:08:22.412111 505 raft_consensus.cc:2454] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P f595c4d2058a4ef087b21d9156cc1c3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb5666afb66f4232ba329f9c7d63f269 in term 0.
I20260812 03:08:22.412699 32624 leader_election.cc:302] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P fb5666afb66f4232ba329f9c7d63f269 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f595c4d2058a4ef087b21d9156cc1c3d, fb5666afb66f4232ba329f9c7d63f269; no voters:
I20260812 03:08:22.412988 570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:22.409511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:55585 (local address 127.26.250.130:45749)
0812 03:08:22.409643 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:22.409648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:22.409663 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:08:22.409704 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:22.409707 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:22.409737 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:22.409824 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:22.412458 (+ 2634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:22.412462 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:22.412465 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:08:22.412479 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:22.412502 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:08:22.412517 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:22.412591 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:22.412807 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 03:08:22.412852 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 03:08:22.413053 580 raft_consensus.cc:2781] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P fb5666afb66f4232ba329f9c7d63f269 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:08:22.413182 580 raft_consensus.cc:482] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P fb5666afb66f4232ba329f9c7d63f269 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:08:22.413254 580 raft_consensus.cc:3037] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P fb5666afb66f4232ba329f9c7d63f269 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:22.413426 582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:22.408923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:45749 (local address 127.26.250.129:55585)
0812 03:08:22.409470 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:22.409486 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:22.409612 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:22.409865 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:22.409870 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:22.409891 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:22.412188 (+ 2297us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:08:22.412190 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:22.412202 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:08:22.412313 (+ 111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:22.412682 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:22.412688 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:22.412692 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:08:22.413217 (+ 525us) client_negotiation.cc:240] Negotiation successful
0812 03:08:22.413279 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":163,"threads_started":1}
I20260812 03:08:22.414089 367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077439a4cb1d4b2b9a6ccd807e9d23c7" candidate_uuid: "fb5666afb66f4232ba329f9c7d63f269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d03837ac8c440a58057b18c0312e60f" is_pre_election: true
I20260812 03:08:22.414438 367 raft_consensus.cc:2454] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P 0d03837ac8c440a58057b18c0312e60f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb5666afb66f4232ba329f9c7d63f269 in term 0.
I20260812 03:08:22.414788 580 raft_consensus.cc:504] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P fb5666afb66f4232ba329f9c7d63f269 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f595c4d2058a4ef087b21d9156cc1c3d" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 33851 } } peers { permanent_uuid: "fb5666afb66f4232ba329f9c7d63f269" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34675 } } peers { permanent_uuid: "0d03837ac8c440a58057b18c0312e60f" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45749 } }
I20260812 03:08:22.415215 580 leader_election.cc:288] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P fb5666afb66f4232ba329f9c7d63f269 [CANDIDATE]: Term 1 election: Requested vote from peers f595c4d2058a4ef087b21d9156cc1c3d (127.26.250.131:33851), 0d03837ac8c440a58057b18c0312e60f (127.26.250.130:45749)
I20260812 03:08:22.415531 505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077439a4cb1d4b2b9a6ccd807e9d23c7" candidate_uuid: "fb5666afb66f4232ba329f9c7d63f269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f595c4d2058a4ef087b21d9156cc1c3d"
I20260812 03:08:22.415686 367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077439a4cb1d4b2b9a6ccd807e9d23c7" candidate_uuid: "fb5666afb66f4232ba329f9c7d63f269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d03837ac8c440a58057b18c0312e60f"
I20260812 03:08:22.415668 505 raft_consensus.cc:3037] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P f595c4d2058a4ef087b21d9156cc1c3d [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:22.415829 367 raft_consensus.cc:3037] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P 0d03837ac8c440a58057b18c0312e60f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:22.416940 367 raft_consensus.cc:2454] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P 0d03837ac8c440a58057b18c0312e60f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb5666afb66f4232ba329f9c7d63f269 in term 1.
I20260812 03:08:22.416951 505 raft_consensus.cc:2454] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P f595c4d2058a4ef087b21d9156cc1c3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb5666afb66f4232ba329f9c7d63f269 in term 1.
I20260812 03:08:22.417264 32623 leader_election.cc:302] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P fb5666afb66f4232ba329f9c7d63f269 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d03837ac8c440a58057b18c0312e60f, fb5666afb66f4232ba329f9c7d63f269; no voters:
I20260812 03:08:22.417451 580 raft_consensus.cc:2781] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P fb5666afb66f4232ba329f9c7d63f269 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:22.417569 580 raft_consensus.cc:686] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P fb5666afb66f4232ba329f9c7d63f269 [term 1 LEADER]: Becoming Leader. State: Replica: fb5666afb66f4232ba329f9c7d63f269, State: Running, Role: LEADER
I20260812 03:08:22.417883 580 consensus_queue.cc:237] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P fb5666afb66f4232ba329f9c7d63f269 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f595c4d2058a4ef087b21d9156cc1c3d" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 33851 } } peers { permanent_uuid: "fb5666afb66f4232ba329f9c7d63f269" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34675 } } peers { permanent_uuid: "0d03837ac8c440a58057b18c0312e60f" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45749 } }
I20260812 03:08:22.420935 32543 catalog_manager.cc:5495] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P fb5666afb66f4232ba329f9c7d63f269 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb5666afb66f4232ba329f9c7d63f269 (127.26.250.129). New cstate: current_term: 1 leader_uuid: "fb5666afb66f4232ba329f9c7d63f269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f595c4d2058a4ef087b21d9156cc1c3d" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 33851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb5666afb66f4232ba329f9c7d63f269" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d03837ac8c440a58057b18c0312e60f" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45749 } health_report { overall_health: UNKNOWN } } }
I20260812 03:08:22.432174 571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:22.430565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57404 (local address 127.26.250.129:34675)
0812 03:08:22.430722 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:22.430727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:22.430744 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:08:22.430888 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:22.430891 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:22.430960 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:22.431054 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:22.431714 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:22.431718 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:22.431720 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:08:22.431733 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:22.431754 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:08:22.431770 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:22.431894 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:22.431978 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 03:08:22.432019 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 03:08:22.438237 367 raft_consensus.cc:1261] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P 0d03837ac8c440a58057b18c0312e60f [term 1 FOLLOWER]: Refusing update from remote peer fb5666afb66f4232ba329f9c7d63f269: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:22.438237 505 raft_consensus.cc:1261] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P f595c4d2058a4ef087b21d9156cc1c3d [term 1 FOLLOWER]: Refusing update from remote peer fb5666afb66f4232ba329f9c7d63f269: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:22.439018 578 consensus_queue.cc:1035] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P fb5666afb66f4232ba329f9c7d63f269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d03837ac8c440a58057b18c0312e60f" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:22.439194 580 consensus_queue.cc:1035] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P fb5666afb66f4232ba329f9c7d63f269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f595c4d2058a4ef087b21d9156cc1c3d" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 33851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:22.449391 586 mvcc.cc:205] Tried to move back new op lower bound from 7317520803579490304 to 7317520803506671616. Current Snapshot: MvccSnapshot[applied={T|T < 7317520803579490304}]
I20260812 03:08:22.451303 587 mvcc.cc:205] Tried to move back new op lower bound from 7317520803579490304 to 7317520803506671616. Current Snapshot: MvccSnapshot[applied={T|T < 7317520803579490304}]
I20260812 03:08:22.455632 32544 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:22.455796 32544 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:22.458617 32544 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b776008ef5ad49e2a630ecde7c6aae79: Sending DeleteTablet for 3 replicas of tablet 077439a4cb1d4b2b9a6ccd807e9d23c7
I20260812 03:08:22.459563 343 tablet_service.cc:1514] Processing DeleteTablet for tablet 077439a4cb1d4b2b9a6ccd807e9d23c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:22 UTC) from {username='slave'} at 127.0.0.1:37920
I20260812 03:08:22.459625 485 tablet_service.cc:1514] Processing DeleteTablet for tablet 077439a4cb1d4b2b9a6ccd807e9d23c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:22 UTC) from {username='slave'} at 127.0.0.1:41028
I20260812 03:08:22.459681 32673 tablet_service.cc:1514] Processing DeleteTablet for tablet 077439a4cb1d4b2b9a6ccd807e9d23c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:22 UTC) from {username='slave'} at 127.0.0.1:57392
I20260812 03:08:22.460319 594 tablet_replica.cc:330] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P 0d03837ac8c440a58057b18c0312e60f: stopping tablet replica
I20260812 03:08:22.460353 595 tablet_replica.cc:330] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P f595c4d2058a4ef087b21d9156cc1c3d: stopping tablet replica
I20260812 03:08:22.460719 594 raft_consensus.cc:2229] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P 0d03837ac8c440a58057b18c0312e60f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:22.460750 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 32608
I20260812 03:08:22.461081 594 raft_consensus.cc:2258] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P 0d03837ac8c440a58057b18c0312e60f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:22.463541 594 ts_tablet_manager.cc:1905] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P 0d03837ac8c440a58057b18c0312e60f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:22.463604 595 raft_consensus.cc:2229] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P f595c4d2058a4ef087b21d9156cc1c3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:22.463953 595 raft_consensus.cc:2258] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P f595c4d2058a4ef087b21d9156cc1c3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:22.465364 595 ts_tablet_manager.cc:1905] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P f595c4d2058a4ef087b21d9156cc1c3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:22.467811 595 ts_tablet_manager.cc:1918] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P f595c4d2058a4ef087b21d9156cc1c3d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:08:22.467878 594 ts_tablet_manager.cc:1918] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P 0d03837ac8c440a58057b18c0312e60f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:08:22.467942 595 log.cc:1196] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P f595c4d2058a4ef087b21d9156cc1c3d: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/077439a4cb1d4b2b9a6ccd807e9d23c7
I20260812 03:08:22.468008 594 log.cc:1196] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P 0d03837ac8c440a58057b18c0312e60f: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786504075277362-27626-0/minicluster-data/ts-1/wal/wals/077439a4cb1d4b2b9a6ccd807e9d23c7
I20260812 03:08:22.468277 595 ts_tablet_manager.cc:1939] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P f595c4d2058a4ef087b21d9156cc1c3d: Deleting consensus metadata
I20260812 03:08:22.468396 594 ts_tablet_manager.cc:1939] T 077439a4cb1d4b2b9a6ccd807e9d23c7 P 0d03837ac8c440a58057b18c0312e60f: Deleting consensus metadata
I20260812 03:08:22.469609 32529 catalog_manager.cc:4841] TS f595c4d2058a4ef087b21d9156cc1c3d (127.26.250.131:33851): tablet 077439a4cb1d4b2b9a6ccd807e9d23c7 (table test-table [id=e0dd368ecc97445184a425831b0724df]) successfully deleted
I20260812 03:08:22.469873 32528 catalog_manager.cc:4841] TS 0d03837ac8c440a58057b18c0312e60f (127.26.250.130:45749): tablet 077439a4cb1d4b2b9a6ccd807e9d23c7 (table test-table [id=e0dd368ecc97445184a425831b0724df]) successfully deleted
W20260812 03:08:22.471998 32528 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.250.129:34675 (error 108)
I20260812 03:08:22.473048 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 32747
I20260812 03:08:22.473320 569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:22.472928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:34675 (local address 127.0.0.1:57420)
0812 03:08:22.473108 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:22.473204 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.129:34675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260812 03:08:22.475013 32528 catalog_manager.cc:4584] TS fb5666afb66f4232ba329f9c7d63f269 (127.26.250.129:34675): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 077439a4cb1d4b2b9a6ccd807e9d23c7: Network error: Client connection negotiation failed: client connection to 127.26.250.129:34675: connect: Connection refused (error 111)
I20260812 03:08:22.482923 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 420
I20260812 03:08:22.493016 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 32512
2026-08-12T03:08:22Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1054 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 03:08:22.518733 27626 mini_oidc.cc:152] Starting JWKS server
I20260812 03:08:22.520674 27626 webserver.cc:460] Webserver started at https://127.0.0.1:37531/ using document root <none> and password file <none>
I20260812 03:08:22.520896 27626 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 03:08:22.521235 27626 webserver.cc:460] Webserver started at http://127.0.0.1:37273/ using document root <none> and password file <none>
2026-08-12T03:08:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:08:22Z Disabled control of system clock
I20260812 03:08:22.532989 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:37097
--webserver_interface=127.26.250.190
--webserver_port=0
--builtin_ntp_servers=127.26.250.148:39559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:37097
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37531/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504075277362-27626-0/testchainca.cert with env {}
W20260812 03:08:22.666108 605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:22.666486 605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:22.666535 605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:22.671531 605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:08:22.671614 605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:22.671633 605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:22.671653 605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:08:22.671670 605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 03:08:22.671689 605 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 03:08:22.671705 605 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37531/jwks.json
I20260812 03:08:22.677953 605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:39559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:37097
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:37097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37531/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504075277362-27626-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:22.679116 605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:22.680240 605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:22.689565 614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:22.689551 610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:22.689522 611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:22.689836 605 server_base.cc:1034] running on GCE node
I20260812 03:08:22.690591 605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:22.691807 605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:22.692987 605 hybrid_clock.cc:648] HybridClock initialized: now 1786504102692974 us; error 31 us; skew 500 ppm
I20260812 03:08:22.695140 605 webserver.cc:460] Webserver started at http://127.26.250.190:42255/ using document root <none> and password file <none>
I20260812 03:08:22.695762 605 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:22.695827 605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:22.696065 605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:22.698212 605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "c8f30ecc97194c53adb7f44ba830e44d"
format_stamp: "Formatted at 2026-08-12 03:08:22 on dist-test-slave-zpfg"
server_key: "4cb318cfa345a3eb7c0efccdfe4f1cd9"
server_key_iv: "f5e333b50abf6f8b2649e70dee8b27cf"
server_key_version: "encryptionkey@0"
I20260812 03:08:22.698906 605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "c8f30ecc97194c53adb7f44ba830e44d"
format_stamp: "Formatted at 2026-08-12 03:08:22 on dist-test-slave-zpfg"
server_key: "4cb318cfa345a3eb7c0efccdfe4f1cd9"
server_key_iv: "f5e333b50abf6f8b2649e70dee8b27cf"
server_key_version: "encryptionkey@0"
I20260812 03:08:22.703050 605 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:08:22.705680 620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:22.707310 605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:08:22.707466 605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "c8f30ecc97194c53adb7f44ba830e44d"
format_stamp: "Formatted at 2026-08-12 03:08:22 on dist-test-slave-zpfg"
server_key: "4cb318cfa345a3eb7c0efccdfe4f1cd9"
server_key_iv: "f5e333b50abf6f8b2649e70dee8b27cf"
server_key_version: "encryptionkey@0"
I20260812 03:08:22.707593 605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:22.730786 605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:22.731556 605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:22.731781 605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:22.739214 605 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:37097
I20260812 03:08:22.739318 673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:37097 every 8 connection(s)
I20260812 03:08:22.740249 605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:08:22.740998 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 605
I20260812 03:08:22.741295 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:08:22.741627 27626 external_mini_cluster.cc:1428] Setting key 669932e5896f89c15624d6e7d46536f3
I20260812 03:08:22.744401 674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:22.751734 674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8f30ecc97194c53adb7f44ba830e44d: Bootstrap starting.
I20260812 03:08:22.752803 677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:22.743087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39026 (local address 127.26.250.190:37097)
0812 03:08:22.743823 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:22.743845 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:22.743891 (+ 46us) server_negotiation.cc:407] Connection header received
0812 03:08:22.744829 (+ 938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:22.744854 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:22.745404 (+ 550us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:22.745789 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:22.746263 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:22.747808 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:22.748779 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:22.749477 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:22.749657 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:22.750638 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:22.750647 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:22.750665 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:08:22.751199 (+ 534us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:22.751274 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:08:22.751300 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:22.751401 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:22.751546 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 03:08:22.751829 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":127,"threads_started":1}
I20260812 03:08:22.754297 674 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8f30ecc97194c53adb7f44ba830e44d: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:22.755213 674 log.cc:824] T 00000000000000000000000000000000 P c8f30ecc97194c53adb7f44ba830e44d: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:22.755862 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:08:22.757496 674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8f30ecc97194c53adb7f44ba830e44d: No bootstrap required, opened a new log
I20260812 03:08:22.761066 674 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8f30ecc97194c53adb7f44ba830e44d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f30ecc97194c53adb7f44ba830e44d" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 37097 } }
I20260812 03:08:22.761297 674 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8f30ecc97194c53adb7f44ba830e44d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:22.761375 674 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8f30ecc97194c53adb7f44ba830e44d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8f30ecc97194c53adb7f44ba830e44d, State: Initialized, Role: FOLLOWER
I20260812 03:08:22.761828 674 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8f30ecc97194c53adb7f44ba830e44d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f30ecc97194c53adb7f44ba830e44d" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 37097 } }
I20260812 03:08:22.761983 674 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8f30ecc97194c53adb7f44ba830e44d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:08:22.762051 674 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8f30ecc97194c53adb7f44ba830e44d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:08:22.762176 674 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8f30ecc97194c53adb7f44ba830e44d [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:22.763185 674 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8f30ecc97194c53adb7f44ba830e44d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f30ecc97194c53adb7f44ba830e44d" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 37097 } }
I20260812 03:08:22.763540 674 leader_election.cc:302] T 00000000000000000000000000000000 P c8f30ecc97194c53adb7f44ba830e44d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8f30ecc97194c53adb7f44ba830e44d; no voters:
I20260812 03:08:22.763899 674 leader_election.cc:288] T 00000000000000000000000000000000 P c8f30ecc97194c53adb7f44ba830e44d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:08:22.764015 680 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8f30ecc97194c53adb7f44ba830e44d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:22.764290 680 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8f30ecc97194c53adb7f44ba830e44d [term 1 LEADER]: Becoming Leader. State: Replica: c8f30ecc97194c53adb7f44ba830e44d, State: Running, Role: LEADER
I20260812 03:08:22.764616 680 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8f30ecc97194c53adb7f44ba830e44d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f30ecc97194c53adb7f44ba830e44d" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 37097 } }
I20260812 03:08:22.765174 674 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8f30ecc97194c53adb7f44ba830e44d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:08:22.766445 681 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8f30ecc97194c53adb7f44ba830e44d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8f30ecc97194c53adb7f44ba830e44d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f30ecc97194c53adb7f44ba830e44d" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 37097 } } }
I20260812 03:08:22.766592 681 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8f30ecc97194c53adb7f44ba830e44d [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:22.766932 682 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8f30ecc97194c53adb7f44ba830e44d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8f30ecc97194c53adb7f44ba830e44d. Latest consensus state: current_term: 1 leader_uuid: "c8f30ecc97194c53adb7f44ba830e44d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8f30ecc97194c53adb7f44ba830e44d" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 37097 } } }
I20260812 03:08:22.767048 682 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8f30ecc97194c53adb7f44ba830e44d [sys.catalog]: This master's current role is: LEADER
W20260812 03:08:22.769968 696 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c8f30ecc97194c53adb7f44ba830e44d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:08:22.770069 696 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:08:22.770161 690 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:08:22.770931 690 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:08:22.779536 690 catalog_manager.cc:1347] Generated new cluster ID: 69c94ef2f24245c481582ed32bc55ce5
I20260812 03:08:22.779621 690 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:08:22.807783 690 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:08:22.808845 690 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:08:22.823118 690 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8f30ecc97194c53adb7f44ba830e44d: Generated new TSK 0
I20260812 03:08:22.823984 690 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 03:08:22.863242 708 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 03:08:22.863467 708 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 03:08:22.864107 677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:22.846366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39040 (local address 127.26.250.190:37097)
0812 03:08:22.846507 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:22.846513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:22.846763 (+ 250us) server_negotiation.cc:407] Connection header received
0812 03:08:22.847005 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:22.847007 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:22.847061 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:22.847153 (+ 92us) server_negotiation.cc:226] Negotiated authn=JWT
0812 03:08:22.847469 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:22.847984 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:22.849248 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:22.849493 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:22.849575 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:22.849983 (+ 408us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 03:08:22.863725 (+ 13742us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37531/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 03:08:22.863988 (+ 263us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39040: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37531/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":46}
W20260812 03:08:22.864199 677 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39040: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37531/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 03:08:22.865942 707 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:22.846228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:37097 (local address 127.0.0.1:39040)
0812 03:08:22.846653 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:22.846671 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:22.846824 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:22.847201 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:22.847204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:22.847213 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0812 03:08:22.847351 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:22.847358 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:22.848118 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:22.848122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:22.849118 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:22.849128 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:22.849616 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:22.849620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:22.849725 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:22.849782 (+ 57us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 03:08:22.864014 (+ 14232us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37531/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 03:08:22.864309 (+ 295us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.250.190:37097: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37531/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":155,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.26.250.190:37097: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37531/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 03:08:22.866757 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 605
2026-08-12T03:08:22Z chronyd exiting
I20260812 03:08:22.907488 27626 test_util.cc:175] -----------------------------------------------
I20260812 03:08:22.907617 27626 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786504075277362-27626-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (391 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 03:08:22.908842 27626 mini_oidc.cc:152] Starting JWKS server
I20260812 03:08:22.909767 27626 webserver.cc:460] Webserver started at https://127.0.0.1:33869/ using document root <none> and password file <none>
I20260812 03:08:22.909921 27626 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 03:08:22.910248 27626 webserver.cc:460] Webserver started at http://127.0.0.1:40391/ using document root <none> and password file <none>
2026-08-12T03:08:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:08:22Z Disabled control of system clock
I20260812 03:08:22.921147 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:35609
--webserver_interface=127.26.250.190
--webserver_port=0
--builtin_ntp_servers=127.26.250.148:43753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:35609
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33869/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504075277362-27626-0/testchainca.cert with env {}
W20260812 03:08:23.043628 714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:23.043919 714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:23.043963 714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:23.048028 714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:08:23.048094 714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:23.048112 714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:23.048133 714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:08:23.048154 714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 03:08:23.048172 714 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 03:08:23.048185 714 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33869/jwks.json
I20260812 03:08:23.053704 714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:43753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:35609
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:35609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33869/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504075277362-27626-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:23.054812 714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:23.055830 714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:23.065359 720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:23.065340 719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:23.065338 722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:23.065667 714 server_base.cc:1034] running on GCE node
I20260812 03:08:23.066248 714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:23.067466 714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:23.068645 714 hybrid_clock.cc:648] HybridClock initialized: now 1786504103068611 us; error 45 us; skew 500 ppm
I20260812 03:08:23.074930 714 webserver.cc:460] Webserver started at http://127.26.250.190:34935/ using document root <none> and password file <none>
I20260812 03:08:23.075552 714 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:23.075627 714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:23.075809 714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:23.077991 714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "ece43b34fa8e4d69817839a5f734b7c1"
format_stamp: "Formatted at 2026-08-12 03:08:23 on dist-test-slave-zpfg"
server_key: "3399aa54478074e02075575028a84b69"
server_key_iv: "d3813bc796d6b0c5f4fe3c14137f5e57"
server_key_version: "encryptionkey@0"
I20260812 03:08:23.078632 714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "ece43b34fa8e4d69817839a5f734b7c1"
format_stamp: "Formatted at 2026-08-12 03:08:23 on dist-test-slave-zpfg"
server_key: "3399aa54478074e02075575028a84b69"
server_key_iv: "d3813bc796d6b0c5f4fe3c14137f5e57"
server_key_version: "encryptionkey@0"
I20260812 03:08:23.082635 714 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 03:08:23.085291 728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:23.086858 714 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:08:23.087010 714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "ece43b34fa8e4d69817839a5f734b7c1"
format_stamp: "Formatted at 2026-08-12 03:08:23 on dist-test-slave-zpfg"
server_key: "3399aa54478074e02075575028a84b69"
server_key_iv: "d3813bc796d6b0c5f4fe3c14137f5e57"
server_key_version: "encryptionkey@0"
I20260812 03:08:23.087129 714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:23.105546 714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:23.106400 714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:23.106619 714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:23.114326 714 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:35609
I20260812 03:08:23.114333 782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:35609 every 8 connection(s)
I20260812 03:08:23.115496 714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:08:23.118112 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 714
I20260812 03:08:23.118271 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:08:23.118637 27626 external_mini_cluster.cc:1428] Setting key 19b3807e6daa5eca0a5f7d7a02826143
I20260812 03:08:23.119094 783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:23.126533 783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ece43b34fa8e4d69817839a5f734b7c1: Bootstrap starting.
I20260812 03:08:23.129426 786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:23.120214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53984 (local address 127.26.250.190:35609)
0812 03:08:23.120901 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:23.120918 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:23.120962 (+ 44us) server_negotiation.cc:407] Connection header received
0812 03:08:23.121774 (+ 812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:23.121794 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:23.122328 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:23.122723 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:23.123103 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:23.124102 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:23.125463 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:23.125950 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:23.126084 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:23.127238 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:23.127243 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:23.127256 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:08:23.127831 (+ 575us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:23.127892 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:08:23.127916 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:23.128004 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:23.128257 (+ 253us) server_negotiation.cc:299] Negotiation successful
0812 03:08:23.128609 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":163,"threads_started":1}
I20260812 03:08:23.129549 783 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ece43b34fa8e4d69817839a5f734b7c1: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:23.130390 783 log.cc:824] T 00000000000000000000000000000000 P ece43b34fa8e4d69817839a5f734b7c1: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:23.132826 783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ece43b34fa8e4d69817839a5f734b7c1: No bootstrap required, opened a new log
I20260812 03:08:23.132965 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:08:23.136896 783 raft_consensus.cc:348] T 00000000000000000000000000000000 P ece43b34fa8e4d69817839a5f734b7c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ece43b34fa8e4d69817839a5f734b7c1" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 35609 } }
I20260812 03:08:23.137315 783 raft_consensus.cc:374] T 00000000000000000000000000000000 P ece43b34fa8e4d69817839a5f734b7c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:23.137432 783 raft_consensus.cc:729] T 00000000000000000000000000000000 P ece43b34fa8e4d69817839a5f734b7c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ece43b34fa8e4d69817839a5f734b7c1, State: Initialized, Role: FOLLOWER
I20260812 03:08:23.137979 783 consensus_queue.cc:260] T 00000000000000000000000000000000 P ece43b34fa8e4d69817839a5f734b7c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ece43b34fa8e4d69817839a5f734b7c1" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 35609 } }
I20260812 03:08:23.138162 783 raft_consensus.cc:388] T 00000000000000000000000000000000 P ece43b34fa8e4d69817839a5f734b7c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:08:23.138236 783 raft_consensus.cc:482] T 00000000000000000000000000000000 P ece43b34fa8e4d69817839a5f734b7c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:08:23.138398 783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ece43b34fa8e4d69817839a5f734b7c1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:23.139483 783 raft_consensus.cc:504] T 00000000000000000000000000000000 P ece43b34fa8e4d69817839a5f734b7c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ece43b34fa8e4d69817839a5f734b7c1" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 35609 } }
I20260812 03:08:23.139886 783 leader_election.cc:302] T 00000000000000000000000000000000 P ece43b34fa8e4d69817839a5f734b7c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ece43b34fa8e4d69817839a5f734b7c1; no voters:
I20260812 03:08:23.140255 783 leader_election.cc:288] T 00000000000000000000000000000000 P ece43b34fa8e4d69817839a5f734b7c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:08:23.140421 788 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ece43b34fa8e4d69817839a5f734b7c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:23.140690 788 raft_consensus.cc:686] T 00000000000000000000000000000000 P ece43b34fa8e4d69817839a5f734b7c1 [term 1 LEADER]: Becoming Leader. State: Replica: ece43b34fa8e4d69817839a5f734b7c1, State: Running, Role: LEADER
I20260812 03:08:23.141004 788 consensus_queue.cc:237] T 00000000000000000000000000000000 P ece43b34fa8e4d69817839a5f734b7c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ece43b34fa8e4d69817839a5f734b7c1" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 35609 } }
I20260812 03:08:23.141702 783 sys_catalog.cc:564] T 00000000000000000000000000000000 P ece43b34fa8e4d69817839a5f734b7c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:08:23.142956 789 sys_catalog.cc:455] T 00000000000000000000000000000000 P ece43b34fa8e4d69817839a5f734b7c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ece43b34fa8e4d69817839a5f734b7c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ece43b34fa8e4d69817839a5f734b7c1" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 35609 } } }
I20260812 03:08:23.143013 790 sys_catalog.cc:455] T 00000000000000000000000000000000 P ece43b34fa8e4d69817839a5f734b7c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ece43b34fa8e4d69817839a5f734b7c1. Latest consensus state: current_term: 1 leader_uuid: "ece43b34fa8e4d69817839a5f734b7c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ece43b34fa8e4d69817839a5f734b7c1" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 35609 } } }
I20260812 03:08:23.143105 790 sys_catalog.cc:458] T 00000000000000000000000000000000 P ece43b34fa8e4d69817839a5f734b7c1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:23.143105 789 sys_catalog.cc:458] T 00000000000000000000000000000000 P ece43b34fa8e4d69817839a5f734b7c1 [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:23.144773 798 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:08:23.147635 798 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:08:23.153354 798 catalog_manager.cc:1347] Generated new cluster ID: 4f6852f8413f463c92fc570021f6d228
I20260812 03:08:23.153435 798 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:08:23.175309 798 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:08:23.176420 798 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:08:23.182631 798 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ece43b34fa8e4d69817839a5f734b7c1: Generated new TSK 0
I20260812 03:08:23.183451 798 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 03:08:23.216802 814 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 03:08:23.216912 814 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 03:08:23.217783 786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:23.197711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53996 (local address 127.26.250.190:35609)
0812 03:08:23.197862 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:23.197868 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:23.198095 (+ 227us) server_negotiation.cc:407] Connection header received
0812 03:08:23.198327 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:23.198330 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:23.198442 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:23.198564 (+ 122us) server_negotiation.cc:226] Negotiated authn=JWT
0812 03:08:23.198831 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:23.199383 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:23.200318 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:23.200561 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:23.200699 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:23.201052 (+ 353us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 03:08:23.217273 (+ 16221us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33869/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 03:08:23.217603 (+ 330us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53996: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33869/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":46}
W20260812 03:08:23.217878 786 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53996: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33869/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260812 03:08:23.217937 813 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:23.197572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:35609 (local address 127.0.0.1:53996)
0812 03:08:23.197979 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:23.197995 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:23.198166 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:23.198564 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:23.198567 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:23.198576 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0812 03:08:23.198704 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:23.198710 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:23.199520 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:23.199524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:23.200189 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:23.200198 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:23.200718 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:23.200722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:23.200818 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:23.200842 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 03:08:23.217509 (+ 16667us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33869/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 03:08:23.217710 (+ 201us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.250.190:35609: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33869/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":223,"spinlock_wait_cycles":80256,"thread_start_us":118,"threads_started":1}
I20260812 03:08:23.218384 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 714
2026-08-12T03:08:23Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (352 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 03:08:23.261406 27626 mini_oidc.cc:152] Starting JWKS server
I20260812 03:08:23.262396 27626 webserver.cc:460] Webserver started at https://127.0.0.1:35809/ using document root <none> and password file <none>
I20260812 03:08:23.262506 27626 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 03:08:23.262828 27626 webserver.cc:460] Webserver started at http://127.0.0.1:34905/ using document root <none> and password file <none>
2026-08-12T03:08:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:08:23Z Disabled control of system clock
I20260812 03:08:23.274555 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:38659
--webserver_interface=127.26.250.190
--webserver_port=0
--builtin_ntp_servers=127.26.250.148:45733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:38659
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35809/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 03:08:23.398464 822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:23.398793 822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:23.398836 822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:23.403016 822 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:08:23.403084 822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:23.403100 822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:23.403117 822 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:08:23.403132 822 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 03:08:23.403151 822 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 03:08:23.403164 822 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35809/jwks.json
I20260812 03:08:23.408016 822 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:45733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:38659
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:38659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35809/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:23.409170 822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:23.410663 822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:08:23.419703 822 server_base.cc:1034] running on GCE node
W20260812 03:08:23.419984 828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:23.420037 827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:23.420037 831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:23.420799 822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:23.421990 822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:23.423167 822 hybrid_clock.cc:648] HybridClock initialized: now 1786504103423101 us; error 82 us; skew 500 ppm
I20260812 03:08:23.425436 822 webserver.cc:460] Webserver started at http://127.26.250.190:45273/ using document root <none> and password file <none>
I20260812 03:08:23.426065 822 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:23.426131 822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:23.426436 822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:23.428682 822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "782bc1bd4625429d9b2dc65a99f60e1f"
format_stamp: "Formatted at 2026-08-12 03:08:23 on dist-test-slave-zpfg"
server_key: "b6b697c3573739c35f759cb99678942e"
server_key_iv: "6c84b691da331e31d3e36607751d4949"
server_key_version: "encryptionkey@0"
I20260812 03:08:23.429312 822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "782bc1bd4625429d9b2dc65a99f60e1f"
format_stamp: "Formatted at 2026-08-12 03:08:23 on dist-test-slave-zpfg"
server_key: "b6b697c3573739c35f759cb99678942e"
server_key_iv: "6c84b691da331e31d3e36607751d4949"
server_key_version: "encryptionkey@0"
I20260812 03:08:23.433279 822 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260812 03:08:23.436060 837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:23.437433 822 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:08:23.437590 822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "782bc1bd4625429d9b2dc65a99f60e1f"
format_stamp: "Formatted at 2026-08-12 03:08:23 on dist-test-slave-zpfg"
server_key: "b6b697c3573739c35f759cb99678942e"
server_key_iv: "6c84b691da331e31d3e36607751d4949"
server_key_version: "encryptionkey@0"
I20260812 03:08:23.437717 822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:23.455873 822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:23.456717 822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:23.456946 822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:23.464892 822 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:38659
I20260812 03:08:23.464910 892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:38659 every 8 connection(s)
I20260812 03:08:23.465921 822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:08:23.469013 893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:23.472415 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 822
I20260812 03:08:23.472529 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:08:23.472828 27626 external_mini_cluster.cc:1428] Setting key 9c9cbde97d1d13e9755fb693bc52be04
I20260812 03:08:23.476101 893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 782bc1bd4625429d9b2dc65a99f60e1f: Bootstrap starting.
I20260812 03:08:23.478566 893 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 782bc1bd4625429d9b2dc65a99f60e1f: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:23.479568 893 log.cc:824] T 00000000000000000000000000000000 P 782bc1bd4625429d9b2dc65a99f60e1f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:23.482189 893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 782bc1bd4625429d9b2dc65a99f60e1f: No bootstrap required, opened a new log
I20260812 03:08:23.483781 897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:23.474359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55396 (local address 127.26.250.190:38659)
0812 03:08:23.475115 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:23.475132 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:23.475180 (+ 48us) server_negotiation.cc:407] Connection header received
0812 03:08:23.475950 (+ 770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:23.475972 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:23.476519 (+ 547us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:23.476908 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:23.477425 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:23.478604 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:23.479972 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:23.480708 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:23.480878 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:23.481754 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:23.481759 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:23.481773 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:08:23.482215 (+ 442us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:23.482313 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:08:23.482394 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:23.482482 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:23.482666 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 03:08:23.483016 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":142,"threads_started":1}
I20260812 03:08:23.486181 893 raft_consensus.cc:348] T 00000000000000000000000000000000 P 782bc1bd4625429d9b2dc65a99f60e1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "782bc1bd4625429d9b2dc65a99f60e1f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 38659 } }
I20260812 03:08:23.486495 893 raft_consensus.cc:374] T 00000000000000000000000000000000 P 782bc1bd4625429d9b2dc65a99f60e1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:23.486575 893 raft_consensus.cc:729] T 00000000000000000000000000000000 P 782bc1bd4625429d9b2dc65a99f60e1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 782bc1bd4625429d9b2dc65a99f60e1f, State: Initialized, Role: FOLLOWER
I20260812 03:08:23.486948 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:08:23.487104 893 consensus_queue.cc:260] T 00000000000000000000000000000000 P 782bc1bd4625429d9b2dc65a99f60e1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "782bc1bd4625429d9b2dc65a99f60e1f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 38659 } }
I20260812 03:08:23.487258 893 raft_consensus.cc:388] T 00000000000000000000000000000000 P 782bc1bd4625429d9b2dc65a99f60e1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:08:23.487318 893 raft_consensus.cc:482] T 00000000000000000000000000000000 P 782bc1bd4625429d9b2dc65a99f60e1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:08:23.487416 893 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 782bc1bd4625429d9b2dc65a99f60e1f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:23.488613 893 raft_consensus.cc:504] T 00000000000000000000000000000000 P 782bc1bd4625429d9b2dc65a99f60e1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "782bc1bd4625429d9b2dc65a99f60e1f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 38659 } }
I20260812 03:08:23.488987 893 leader_election.cc:302] T 00000000000000000000000000000000 P 782bc1bd4625429d9b2dc65a99f60e1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 782bc1bd4625429d9b2dc65a99f60e1f; no voters:
I20260812 03:08:23.489320 893 leader_election.cc:288] T 00000000000000000000000000000000 P 782bc1bd4625429d9b2dc65a99f60e1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:08:23.489480 900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 782bc1bd4625429d9b2dc65a99f60e1f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:23.489729 900 raft_consensus.cc:686] T 00000000000000000000000000000000 P 782bc1bd4625429d9b2dc65a99f60e1f [term 1 LEADER]: Becoming Leader. State: Replica: 782bc1bd4625429d9b2dc65a99f60e1f, State: Running, Role: LEADER
I20260812 03:08:23.490099 900 consensus_queue.cc:237] T 00000000000000000000000000000000 P 782bc1bd4625429d9b2dc65a99f60e1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "782bc1bd4625429d9b2dc65a99f60e1f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 38659 } }
I20260812 03:08:23.490726 893 sys_catalog.cc:564] T 00000000000000000000000000000000 P 782bc1bd4625429d9b2dc65a99f60e1f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:08:23.491971 902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 782bc1bd4625429d9b2dc65a99f60e1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 782bc1bd4625429d9b2dc65a99f60e1f. Latest consensus state: current_term: 1 leader_uuid: "782bc1bd4625429d9b2dc65a99f60e1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "782bc1bd4625429d9b2dc65a99f60e1f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 38659 } } }
I20260812 03:08:23.491971 901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 782bc1bd4625429d9b2dc65a99f60e1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "782bc1bd4625429d9b2dc65a99f60e1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "782bc1bd4625429d9b2dc65a99f60e1f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 38659 } } }
I20260812 03:08:23.492141 901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 782bc1bd4625429d9b2dc65a99f60e1f [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:23.492415 902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 782bc1bd4625429d9b2dc65a99f60e1f [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:23.492681 909 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:08:23.496251 909 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:08:23.502542 909 catalog_manager.cc:1347] Generated new cluster ID: 27a9c35bf3dd45c4a6e5747f4185ac4a
I20260812 03:08:23.502641 909 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:08:23.509488 909 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:08:23.510843 909 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:08:23.523932 909 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 782bc1bd4625429d9b2dc65a99f60e1f: Generated new TSK 0
I20260812 03:08:23.524760 909 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:08:23.543864 897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:23.538529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55408 (local address 127.26.250.190:38659)
0812 03:08:23.538676 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:23.538682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:23.538805 (+ 123us) server_negotiation.cc:407] Connection header received
0812 03:08:23.538942 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:23.538945 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:23.539005 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:23.539128 (+ 123us) server_negotiation.cc:226] Negotiated authn=JWT
0812 03:08:23.539333 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:23.540211 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:23.541101 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:23.541452 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:23.541561 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:23.541849 (+ 288us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 03:08:23.543460 (+ 1611us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35809/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 03:08:23.543703 (+ 243us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55408: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35809/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":68}
W20260812 03:08:23.543947 897 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55408: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35809/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 03:08:23.545473 926 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:23.538403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:38659 (local address 127.0.0.1:55408)
0812 03:08:23.538712 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:23.538726 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:23.538828 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:23.539130 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:23.539132 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:23.539138 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0812 03:08:23.539231 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:23.539235 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:23.540350 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:23.540352 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:23.540986 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:23.540993 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:23.541579 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:23.541582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:23.541646 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:23.541670 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 03:08:23.543635 (+ 1965us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35809/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 03:08:23.545270 (+ 1635us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.250.190:38659: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35809/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":1430,"thread_start_us":127,"threads_started":1}
I20260812 03:08:23.546110 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 822
E20260812 03:08:23.558017 927 webserver.cc:533] Webserver: sslize failed:
E20260812 03:08:23.558163 927 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-12T03:08:23Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (328 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T03:08:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:08:23Z Disabled control of system clock
I20260812 03:08:23.600647 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:46473
--webserver_interface=127.26.250.190
--webserver_port=0
--builtin_ntp_servers=127.26.250.148:40331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:46473
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504075277362-27626-0/testchainca.cert with env {}
W20260812 03:08:23.731482 931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:23.731845 931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:23.731923 931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:23.736173 931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:08:23.736294 931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:23.736322 931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:23.736363 931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:08:23.736418 931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 03:08:23.736457 931 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 03:08:23.736490 931 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 03:08:23.742187 931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:40331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:46473
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:46473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504075277362-27626-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:23.743417 931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:23.744517 931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:23.754053 936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:23.754033 937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:23.754007 939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:23.754281 931 server_base.cc:1034] running on GCE node
I20260812 03:08:23.755129 931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:23.756389 931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:23.757620 931 hybrid_clock.cc:648] HybridClock initialized: now 1786504103757597 us; error 36 us; skew 500 ppm
I20260812 03:08:23.759891 931 webserver.cc:460] Webserver started at http://127.26.250.190:45649/ using document root <none> and password file <none>
I20260812 03:08:23.760520 931 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:23.760619 931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:23.760881 931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:23.763175 931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "95e441aa975a4969b8989ca2e21d6917"
format_stamp: "Formatted at 2026-08-12 03:08:23 on dist-test-slave-zpfg"
server_key: "e730f8de7baa2bf98f0f89c9c5ece851"
server_key_iv: "63aea919cbf576ff0faddc956147a695"
server_key_version: "encryptionkey@0"
I20260812 03:08:23.763799 931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "95e441aa975a4969b8989ca2e21d6917"
format_stamp: "Formatted at 2026-08-12 03:08:23 on dist-test-slave-zpfg"
server_key: "e730f8de7baa2bf98f0f89c9c5ece851"
server_key_iv: "63aea919cbf576ff0faddc956147a695"
server_key_version: "encryptionkey@0"
I20260812 03:08:23.768018 931 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:08:23.770823 945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:23.772207 931 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:08:23.772421 931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "95e441aa975a4969b8989ca2e21d6917"
format_stamp: "Formatted at 2026-08-12 03:08:23 on dist-test-slave-zpfg"
server_key: "e730f8de7baa2bf98f0f89c9c5ece851"
server_key_iv: "63aea919cbf576ff0faddc956147a695"
server_key_version: "encryptionkey@0"
I20260812 03:08:23.772560 931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:23.785890 931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:23.786798 931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:23.787027 931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:23.795190 931 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:46473
I20260812 03:08:23.795197 1002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:46473 every 8 connection(s)
I20260812 03:08:23.796293 931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:08:23.798146 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 931
I20260812 03:08:23.798254 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:08:23.798590 27626 external_mini_cluster.cc:1428] Setting key cd1ad2f4518001d3a525a3e3efc6c27b
I20260812 03:08:23.799615 1003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:23.806433 1003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95e441aa975a4969b8989ca2e21d6917: Bootstrap starting.
I20260812 03:08:23.808527 1006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:23.800105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60916 (local address 127.26.250.190:46473)
0812 03:08:23.800798 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:23.800820 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:23.800863 (+ 43us) server_negotiation.cc:407] Connection header received
0812 03:08:23.801537 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:23.801554 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:23.801962 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:23.802417 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:23.802852 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:23.803851 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:23.804787 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:23.805295 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:23.805474 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:23.806613 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:23.806621 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:23.806640 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:08:23.807167 (+ 527us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:23.807229 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:08:23.807251 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:23.807377 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:23.807499 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 03:08:23.807777 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":151,"threads_started":1}
I20260812 03:08:23.809110 1003 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95e441aa975a4969b8989ca2e21d6917: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:23.809979 1003 log.cc:824] T 00000000000000000000000000000000 P 95e441aa975a4969b8989ca2e21d6917: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:23.811673 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:08:23.812691 1003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95e441aa975a4969b8989ca2e21d6917: No bootstrap required, opened a new log
I20260812 03:08:23.816608 1003 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95e441aa975a4969b8989ca2e21d6917 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95e441aa975a4969b8989ca2e21d6917" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 46473 } }
I20260812 03:08:23.816869 1003 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95e441aa975a4969b8989ca2e21d6917 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:23.816907 1003 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95e441aa975a4969b8989ca2e21d6917 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95e441aa975a4969b8989ca2e21d6917, State: Initialized, Role: FOLLOWER
I20260812 03:08:23.817473 1003 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95e441aa975a4969b8989ca2e21d6917 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95e441aa975a4969b8989ca2e21d6917" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 46473 } }
I20260812 03:08:23.817692 1003 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95e441aa975a4969b8989ca2e21d6917 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:08:23.817780 1003 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95e441aa975a4969b8989ca2e21d6917 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:08:23.817914 1003 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95e441aa975a4969b8989ca2e21d6917 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:23.819046 1003 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95e441aa975a4969b8989ca2e21d6917 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95e441aa975a4969b8989ca2e21d6917" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 46473 } }
I20260812 03:08:23.819428 1003 leader_election.cc:302] T 00000000000000000000000000000000 P 95e441aa975a4969b8989ca2e21d6917 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95e441aa975a4969b8989ca2e21d6917; no voters:
I20260812 03:08:23.819777 1003 leader_election.cc:288] T 00000000000000000000000000000000 P 95e441aa975a4969b8989ca2e21d6917 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:08:23.819970 1008 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95e441aa975a4969b8989ca2e21d6917 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:23.820215 1008 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95e441aa975a4969b8989ca2e21d6917 [term 1 LEADER]: Becoming Leader. State: Replica: 95e441aa975a4969b8989ca2e21d6917, State: Running, Role: LEADER
I20260812 03:08:23.820541 1008 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95e441aa975a4969b8989ca2e21d6917 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95e441aa975a4969b8989ca2e21d6917" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 46473 } }
I20260812 03:08:23.821395 1003 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95e441aa975a4969b8989ca2e21d6917 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:08:23.821859 1009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95e441aa975a4969b8989ca2e21d6917 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95e441aa975a4969b8989ca2e21d6917. Latest consensus state: current_term: 1 leader_uuid: "95e441aa975a4969b8989ca2e21d6917" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95e441aa975a4969b8989ca2e21d6917" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 46473 } } }
I20260812 03:08:23.822009 1009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95e441aa975a4969b8989ca2e21d6917 [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:23.822315 1010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95e441aa975a4969b8989ca2e21d6917 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95e441aa975a4969b8989ca2e21d6917" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95e441aa975a4969b8989ca2e21d6917" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 46473 } } }
I20260812 03:08:23.822463 1010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95e441aa975a4969b8989ca2e21d6917 [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:23.822577 1012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:08:23.825804 1012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:08:23.832551 1012 catalog_manager.cc:1347] Generated new cluster ID: 5bebe48ce1484d4caa4695e9ebf47b11
I20260812 03:08:23.832661 1012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:08:23.846130 1012 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:08:23.847582 1012 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:08:23.864144 1012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95e441aa975a4969b8989ca2e21d6917: Generated new TSK 0
I20260812 03:08:23.865248 1012 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:08:23.982841 1006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:23.884453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60930 (local address 127.26.250.190:46473)
0812 03:08:23.884612 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:23.884618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:23.884778 (+ 160us) server_negotiation.cc:407] Connection header received
0812 03:08:23.884935 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:23.884939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:23.885000 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:23.885132 (+ 132us) server_negotiation.cc:226] Negotiated authn=JWT
0812 03:08:23.885429 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:23.886295 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:23.887294 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:23.887679 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:23.887789 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:23.888168 (+ 379us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 03:08:23.982297 (+ 94129us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 03:08:23.982696 (+ 399us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60930: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":60}
W20260812 03:08:23.982954 1006 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60930: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 03:08:23.983107 1034 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:23.884326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:46473 (local address 127.0.0.1:60930)
0812 03:08:23.884681 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:23.884696 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:23.884805 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:23.885159 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:23.885162 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:23.885169 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0812 03:08:23.885275 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:23.885281 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:23.886478 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:23.886481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:23.887165 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:23.887174 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:23.887847 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:23.887851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:23.887932 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:23.887961 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 03:08:23.982629 (+ 94668us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 03:08:23.982895 (+ 266us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.26.250.190:46473: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":133,"threads_started":1}
I20260812 03:08:23.983773 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 931
2026-08-12T03:08:23Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (416 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (146 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (140 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:08:24 dist-test-slave-zpfg krb5kdc[1047](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:08:24 dist-test-slave-zpfg krb5kdc[1047](info): set up 2 sockets
Aug 12 03:08:24 dist-test-slave-zpfg krb5kdc[1047](info): commencing operation
krb5kdc: starting...
W20260812 03:08:25.375540 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:08:25 dist-test-slave-zpfg krb5kdc[1047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:08:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:08:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:08:25.539888 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:39337
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:37403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:39337
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:25.665977 1062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:25.666342 1062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:25.666460 1062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:25.670809 1062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:08:25.670934 1062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:25.670984 1062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:25.671039 1062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:08:25.671087 1062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:08:25.676934 1062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:37403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:39337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:39337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:25.678215 1062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:25.679354 1062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:25.688901 1070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:25.688930 1068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:25.688930 1067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:25.689146 1062 server_base.cc:1034] running on GCE node
I20260812 03:08:25.690085 1062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:25.691512 1062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:25.692746 1062 hybrid_clock.cc:648] HybridClock initialized: now 1786504105692699 us; error 64 us; skew 500 ppm
Aug 12 03:08:25 dist-test-slave-zpfg krb5kdc[1047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504105, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:25.695990 1062 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:08:25.697067 1062 webserver.cc:460] Webserver started at http://127.26.250.190:46567/ using document root <none> and password file <none>
I20260812 03:08:25.697757 1062 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:25.697856 1062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:25.698109 1062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:25.700485 1062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "443be5fdc3d848c1973dac5b2fd8641e"
format_stamp: "Formatted at 2026-08-12 03:08:25 on dist-test-slave-zpfg"
server_key: "5d3330e88e63dba0000816fa93f9b3ac"
server_key_iv: "52e18568055b1228c216305da0774276"
server_key_version: "encryptionkey@0"
I20260812 03:08:25.701153 1062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "443be5fdc3d848c1973dac5b2fd8641e"
format_stamp: "Formatted at 2026-08-12 03:08:25 on dist-test-slave-zpfg"
server_key: "5d3330e88e63dba0000816fa93f9b3ac"
server_key_iv: "52e18568055b1228c216305da0774276"
server_key_version: "encryptionkey@0"
I20260812 03:08:25.705431 1062 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:08:25.708343 1077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:25.710067 1062 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:08:25.710278 1062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "443be5fdc3d848c1973dac5b2fd8641e"
format_stamp: "Formatted at 2026-08-12 03:08:25 on dist-test-slave-zpfg"
server_key: "5d3330e88e63dba0000816fa93f9b3ac"
server_key_iv: "52e18568055b1228c216305da0774276"
server_key_version: "encryptionkey@0"
I20260812 03:08:25.710454 1062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:25.734812 1062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:25.738639 1062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:25.738942 1062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:25.747609 1062 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:39337
I20260812 03:08:25.747641 1134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:39337 every 8 connection(s)
I20260812 03:08:25.748842 1062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:08:25.752002 1135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:25.757984 1135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 443be5fdc3d848c1973dac5b2fd8641e: Bootstrap starting.
I20260812 03:08:25.758781 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 1062
I20260812 03:08:25.758919 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:08:25.759240 27626 external_mini_cluster.cc:1428] Setting key 77191ac2a449f18a2a223cd0b9d39986
I20260812 03:08:25.760946 1135 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 443be5fdc3d848c1973dac5b2fd8641e: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:25.761768 1135 log.cc:824] T 00000000000000000000000000000000 P 443be5fdc3d848c1973dac5b2fd8641e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:25.764051 1135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 443be5fdc3d848c1973dac5b2fd8641e: No bootstrap required, opened a new log
I20260812 03:08:25.768074 1135 raft_consensus.cc:348] T 00000000000000000000000000000000 P 443be5fdc3d848c1973dac5b2fd8641e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "443be5fdc3d848c1973dac5b2fd8641e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39337 } }
I20260812 03:08:25.768368 1135 raft_consensus.cc:374] T 00000000000000000000000000000000 P 443be5fdc3d848c1973dac5b2fd8641e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:25.768471 1135 raft_consensus.cc:729] T 00000000000000000000000000000000 P 443be5fdc3d848c1973dac5b2fd8641e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 443be5fdc3d848c1973dac5b2fd8641e, State: Initialized, Role: FOLLOWER
Aug 12 03:08:25 dist-test-slave-zpfg krb5kdc[1047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:25.769028 1135 consensus_queue.cc:260] T 00000000000000000000000000000000 P 443be5fdc3d848c1973dac5b2fd8641e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "443be5fdc3d848c1973dac5b2fd8641e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39337 } }
I20260812 03:08:25.769249 1135 raft_consensus.cc:388] T 00000000000000000000000000000000 P 443be5fdc3d848c1973dac5b2fd8641e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:08:25.769335 1135 raft_consensus.cc:482] T 00000000000000000000000000000000 P 443be5fdc3d848c1973dac5b2fd8641e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:08:25.769474 1135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 443be5fdc3d848c1973dac5b2fd8641e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:25.770795 1135 raft_consensus.cc:504] T 00000000000000000000000000000000 P 443be5fdc3d848c1973dac5b2fd8641e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "443be5fdc3d848c1973dac5b2fd8641e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39337 } }
I20260812 03:08:25.771248 1135 leader_election.cc:302] T 00000000000000000000000000000000 P 443be5fdc3d848c1973dac5b2fd8641e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 443be5fdc3d848c1973dac5b2fd8641e; no voters:
I20260812 03:08:25.771698 1135 leader_election.cc:288] T 00000000000000000000000000000000 P 443be5fdc3d848c1973dac5b2fd8641e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:08:25.771795 1140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 443be5fdc3d848c1973dac5b2fd8641e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:25.772037 1140 raft_consensus.cc:686] T 00000000000000000000000000000000 P 443be5fdc3d848c1973dac5b2fd8641e [term 1 LEADER]: Becoming Leader. State: Replica: 443be5fdc3d848c1973dac5b2fd8641e, State: Running, Role: LEADER
I20260812 03:08:25.772431 1140 consensus_queue.cc:237] T 00000000000000000000000000000000 P 443be5fdc3d848c1973dac5b2fd8641e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "443be5fdc3d848c1973dac5b2fd8641e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39337 } }
I20260812 03:08:25.773285 1135 sys_catalog.cc:564] T 00000000000000000000000000000000 P 443be5fdc3d848c1973dac5b2fd8641e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:08:25.774696 1141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 443be5fdc3d848c1973dac5b2fd8641e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "443be5fdc3d848c1973dac5b2fd8641e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "443be5fdc3d848c1973dac5b2fd8641e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39337 } } }
I20260812 03:08:25.774693 1142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 443be5fdc3d848c1973dac5b2fd8641e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 443be5fdc3d848c1973dac5b2fd8641e. Latest consensus state: current_term: 1 leader_uuid: "443be5fdc3d848c1973dac5b2fd8641e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "443be5fdc3d848c1973dac5b2fd8641e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39337 } } }
I20260812 03:08:25.774879 1141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 443be5fdc3d848c1973dac5b2fd8641e [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:25.774892 1142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 443be5fdc3d848c1973dac5b2fd8641e [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:25.775367 1149 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:08:25.776789 1138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:25.760525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42364 (local address 127.26.250.190:39337)
0812 03:08:25.761018 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:25.761035 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:25.761103 (+ 68us) server_negotiation.cc:407] Connection header received
0812 03:08:25.761839 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:25.761859 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:25.762394 (+ 535us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:25.762860 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:25.763788 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:25.764998 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:25.765979 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:25.766647 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:25.766849 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:25.769856 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:25.769862 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:25.769878 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:25.769911 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:25.772076 (+ 2165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:25.772703 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:25.772708 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:25.772713 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:25.772846 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:25.773403 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:25.773410 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:25.773412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:25.774018 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:25.775025 (+ 1007us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:25.775217 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 03:08:25.775607 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":150,"threads_started":1}
I20260812 03:08:25.780988 1149 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:08:25.788240 1149 catalog_manager.cc:1347] Generated new cluster ID: 095e3c6fd8cf4ff69a25248aa2472e76
I20260812 03:08:25.788363 1149 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:08:25.800944 1149 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:08:25.802397 1149 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:08:25.812398 1149 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 443be5fdc3d848c1973dac5b2fd8641e: Generated new TSK 0
I20260812 03:08:25.813246 1149 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.250.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
I20260812 03:08:25.858233 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:0
--local_ip_for_outbound_sockets=127.26.250.129
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39337
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:37403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:25.994561 1162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:25.994890 1162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:25.994942 1162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:25.999611 1162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:25.999698 1162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:25.999763 1162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:08:26.005417 1162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:37403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:26.006728 1162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:26.007794 1162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:26.018687 1170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:26.018656 1167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:26.018692 1168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:26.018991 1162 server_base.cc:1034] running on GCE node
I20260812 03:08:26.019505 1162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:26.020241 1162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:26.021474 1162 hybrid_clock.cc:648] HybridClock initialized: now 1786504106021434 us; error 56 us; skew 500 ppm
Aug 12 03:08:26 dist-test-slave-zpfg krb5kdc[1047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504106, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:26.024683 1162 init.cc:377] Logged in from keytab as kudu/127.26.250.129@KRBTEST.COM (short username kudu)
I20260812 03:08:26.025785 1162 webserver.cc:460] Webserver started at http://127.26.250.129:37997/ using document root <none> and password file <none>
I20260812 03:08:26.026571 1162 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:26.026674 1162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:26.026928 1162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:26.029147 1162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-0/data/instance:
uuid: "40091d55951c46d08322607feeddb305"
format_stamp: "Formatted at 2026-08-12 03:08:26 on dist-test-slave-zpfg"
server_key: "e7cd4e508d9e2374b3348cb6300cb787"
server_key_iv: "08e64d78cfe9e2862bb790d7f13c9452"
server_key_version: "encryptionkey@0"
I20260812 03:08:26.029767 1162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance:
uuid: "40091d55951c46d08322607feeddb305"
format_stamp: "Formatted at 2026-08-12 03:08:26 on dist-test-slave-zpfg"
server_key: "e7cd4e508d9e2374b3348cb6300cb787"
server_key_iv: "08e64d78cfe9e2862bb790d7f13c9452"
server_key_version: "encryptionkey@0"
I20260812 03:08:26.033838 1162 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:08:26.036803 1178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:26.038277 1162 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 03:08:26.038472 1162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "40091d55951c46d08322607feeddb305"
format_stamp: "Formatted at 2026-08-12 03:08:26 on dist-test-slave-zpfg"
server_key: "e7cd4e508d9e2374b3348cb6300cb787"
server_key_iv: "08e64d78cfe9e2862bb790d7f13c9452"
server_key_version: "encryptionkey@0"
I20260812 03:08:26.038611 1162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:26.074898 1162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:26.078994 1162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:26.079248 1162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:26.079977 1162 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:26.080930 1162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:26.081012 1162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:26.081094 1162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:26.081144 1162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:26.091833 1162 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:44569
I20260812 03:08:26.091856 1300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:44569 every 8 connection(s)
I20260812 03:08:26.092916 1162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:08:26.097800 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 1162
I20260812 03:08:26.097975 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance
I20260812 03:08:26.098294 27626 external_mini_cluster.cc:1428] Setting key cde7647aa7b4095e991ea69c1a269dad
Aug 12 03:08:26 dist-test-slave-zpfg krb5kdc[1047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504106, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.250.130@KRBTEST.COM; defaulting to no policy
I20260812 03:08:26.110179 1138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:26.095014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:51875 (local address 127.26.250.190:39337)
0812 03:08:26.095167 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:26.095171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:26.096148 (+ 977us) server_negotiation.cc:407] Connection header received
0812 03:08:26.097636 (+ 1488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:26.097639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:26.097701 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:26.097794 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:26.099435 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:26.099974 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:26.101224 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:26.101472 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:26.101588 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:26.104124 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:26.104128 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:26.104132 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:26.104167 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:26.107847 (+ 3680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:26.108439 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:26.108444 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:26.108446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:26.108538 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:26.108905 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:26.108910 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:26.108912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:26.109130 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:26.109246 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:26.109833 (+ 587us) server_negotiation.cc:299] Negotiation successful
0812 03:08:26.109991 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 03:08:26.111208 1304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:26.095434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39337 (local address 127.26.250.129:51875)
0812 03:08:26.095963 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:26.096012 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:26.097399 (+ 1387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:26.098000 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:26.098010 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:26.098648 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:26.099277 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:26.099301 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:26.100110 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:26.100113 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:26.101086 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:26.101095 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:26.101635 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:26.101639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:26.101824 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:26.102491 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:26.102508 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:26.103949 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:26.107989 (+ 4040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:26.107995 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:26.108014 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:26.108326 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:26.108637 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:26.108641 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:26.108643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:26.108806 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:26.109247 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:26.109255 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:26.109539 (+ 284us) client_negotiation.cc:769] Sending connection context
0812 03:08:26.109780 (+ 241us) client_negotiation.cc:240] Negotiation successful
0812 03:08:26.110148 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":142,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.130@KRBTEST.COM" created.
I20260812 03:08:26.112901 1301 heartbeater.cc:347] Connected to a master server at 127.26.250.190:39337
I20260812 03:08:26.113179 1301 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:26.113734 1301 heartbeater.cc:511] Master 127.26.250.190:39337 requested a full tablet report, sending...
I20260812 03:08:26.115684 1094 ts_manager.cc:194] Registered new tserver with Master: 40091d55951c46d08322607feeddb305 (127.26.250.129:44569)
I20260812 03:08:26.116899 1094 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.129@KRBTEST.COM'} at 127.26.250.129:51875
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
I20260812 03:08:26.130573 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:0
--local_ip_for_outbound_sockets=127.26.250.130
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39337
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:37403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:26.258661 1307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:26.258992 1307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:26.259044 1307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:26.263983 1307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:26.264073 1307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:26.264191 1307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:08:26.270649 1307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:37403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:26.271917 1307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:26.273206 1307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:26.286686 1315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:26.286679 1307 server_base.cc:1034] running on GCE node
W20260812 03:08:26.286614 1312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:26.286614 1313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:26.287618 1307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:26.288360 1307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:26.289597 1307 hybrid_clock.cc:648] HybridClock initialized: now 1786504106289565 us; error 46 us; skew 500 ppm
Aug 12 03:08:26 dist-test-slave-zpfg krb5kdc[1047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504106, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:26.293368 1307 init.cc:377] Logged in from keytab as kudu/127.26.250.130@KRBTEST.COM (short username kudu)
I20260812 03:08:26.294528 1307 webserver.cc:460] Webserver started at http://127.26.250.130:42389/ using document root <none> and password file <none>
I20260812 03:08:26.295305 1307 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:26.295420 1307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:26.295681 1307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:26.298058 1307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-1/data/instance:
uuid: "472087b6600c45eb97316cd7b005e5d3"
format_stamp: "Formatted at 2026-08-12 03:08:26 on dist-test-slave-zpfg"
server_key: "2d90af7a903a0797775ee6908f5a0ed3"
server_key_iv: "877da1cc1b2c28de5783c4626fad2bff"
server_key_version: "encryptionkey@0"
I20260812 03:08:26.298839 1307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance:
uuid: "472087b6600c45eb97316cd7b005e5d3"
format_stamp: "Formatted at 2026-08-12 03:08:26 on dist-test-slave-zpfg"
server_key: "2d90af7a903a0797775ee6908f5a0ed3"
server_key_iv: "877da1cc1b2c28de5783c4626fad2bff"
server_key_version: "encryptionkey@0"
I20260812 03:08:26.303987 1307 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 03:08:26.309058 1322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:26.311299 1307 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 03:08:26.311486 1307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "472087b6600c45eb97316cd7b005e5d3"
format_stamp: "Formatted at 2026-08-12 03:08:26 on dist-test-slave-zpfg"
server_key: "2d90af7a903a0797775ee6908f5a0ed3"
server_key_iv: "877da1cc1b2c28de5783c4626fad2bff"
server_key_version: "encryptionkey@0"
I20260812 03:08:26.311692 1307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:26.326303 1307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:26.330561 1307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:26.330751 1307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:26.331458 1307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:26.332592 1307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:26.332650 1307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:26.332696 1307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:26.332715 1307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:26.343685 1307 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:41553
I20260812 03:08:26.343711 1446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:41553 every 8 connection(s)
I20260812 03:08:26.344684 1307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:08:26.349607 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 1307
I20260812 03:08:26.349788 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance
I20260812 03:08:26.350165 27626 external_mini_cluster.cc:1428] Setting key 07ba8550ba102dbd5d74ccbaa57024f9
Aug 12 03:08:26 dist-test-slave-zpfg krb5kdc[1047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504106, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:26.363472 1138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:26.346817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:37389 (local address 127.26.250.190:39337)
0812 03:08:26.347003 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:26.347007 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:26.347990 (+ 983us) server_negotiation.cc:407] Connection header received
0812 03:08:26.349642 (+ 1652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:26.349645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:26.349711 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:26.349804 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:26.351642 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:26.352404 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:26.354518 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:26.354989 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:26.355139 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:26.358372 (+ 3233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:26.358377 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:26.358381 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:26.358428 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:26.360604 (+ 2176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:26.361279 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:26.361283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:26.361286 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:26.361406 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:26.361795 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:26.361802 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:26.361805 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:26.362108 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:26.362250 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:26.363098 (+ 848us) server_negotiation.cc:299] Negotiation successful
0812 03:08:26.363272 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
WARNING: no policy specified for kudu/127.26.250.131@KRBTEST.COM; defaulting to no policy
I20260812 03:08:26.364534 1449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:26.347181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39337 (local address 127.26.250.130:37389)
0812 03:08:26.347723 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:26.347773 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:26.349397 (+ 1624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:26.350030 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:26.350042 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:26.350666 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:26.351453 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:26.351470 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:26.352658 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:26.352662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:26.354284 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:26.354299 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:26.355180 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:26.355183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:26.355368 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:26.356065 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:26.356089 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:26.358109 (+ 2020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:26.360827 (+ 2718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:26.360832 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:26.360848 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:26.361149 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:26.361524 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:26.361526 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:26.361527 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:26.361678 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:26.362305 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:26.362313 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:26.362774 (+ 461us) client_negotiation.cc:769] Sending connection context
0812 03:08:26.362981 (+ 207us) client_negotiation.cc:240] Negotiation successful
0812 03:08:26.363439 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":128,"threads_started":1}
I20260812 03:08:26.366078 1447 heartbeater.cc:347] Connected to a master server at 127.26.250.190:39337
I20260812 03:08:26.366539 1447 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.131@KRBTEST.COM" created.
I20260812 03:08:26.367213 1447 heartbeater.cc:511] Master 127.26.250.190:39337 requested a full tablet report, sending...
I20260812 03:08:26.368533 1094 ts_manager.cc:194] Registered new tserver with Master: 472087b6600c45eb97316cd7b005e5d3 (127.26.250.130:41553)
I20260812 03:08:26.369460 1094 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.130@KRBTEST.COM'} at 127.26.250.130:37389
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
I20260812 03:08:26.387454 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:0
--local_ip_for_outbound_sockets=127.26.250.131
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39337
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:37403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:26.512534 1452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:26.512861 1452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:26.512908 1452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:26.517400 1452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:26.517474 1452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:26.517537 1452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:08:26.523046 1452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:37403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:26.524179 1452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:26.525346 1452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:26.536782 1457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:26.536804 1458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:26.536782 1460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:26.537585 1452 server_base.cc:1034] running on GCE node
I20260812 03:08:26.538057 1452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:26.538877 1452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:26.540073 1452 hybrid_clock.cc:648] HybridClock initialized: now 1786504106540038 us; error 58 us; skew 500 ppm
Aug 12 03:08:26 dist-test-slave-zpfg krb5kdc[1047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504106, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:26.543308 1452 init.cc:377] Logged in from keytab as kudu/127.26.250.131@KRBTEST.COM (short username kudu)
I20260812 03:08:26.544410 1452 webserver.cc:460] Webserver started at http://127.26.250.131:33001/ using document root <none> and password file <none>
I20260812 03:08:26.545085 1452 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:26.545207 1452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:26.545481 1452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:26.547930 1452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-2/data/instance:
uuid: "536c614daa9a44498d60d6c76da1a382"
format_stamp: "Formatted at 2026-08-12 03:08:26 on dist-test-slave-zpfg"
server_key: "d342a012f31b56dabbeee8db178a20c4"
server_key_iv: "831dc0850b0bbec612c9b6dee2e95a37"
server_key_version: "encryptionkey@0"
I20260812 03:08:26.548655 1452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance:
uuid: "536c614daa9a44498d60d6c76da1a382"
format_stamp: "Formatted at 2026-08-12 03:08:26 on dist-test-slave-zpfg"
server_key: "d342a012f31b56dabbeee8db178a20c4"
server_key_iv: "831dc0850b0bbec612c9b6dee2e95a37"
server_key_version: "encryptionkey@0"
I20260812 03:08:26.553130 1452 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:08:26.556202 1468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:26.557694 1452 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 03:08:26.557864 1452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "536c614daa9a44498d60d6c76da1a382"
format_stamp: "Formatted at 2026-08-12 03:08:26 on dist-test-slave-zpfg"
server_key: "d342a012f31b56dabbeee8db178a20c4"
server_key_iv: "831dc0850b0bbec612c9b6dee2e95a37"
server_key_version: "encryptionkey@0"
I20260812 03:08:26.557996 1452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:26.575697 1452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:26.579732 1452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:26.579938 1452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:26.580521 1452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:26.581455 1452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:26.581533 1452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:26.581609 1452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:26.581660 1452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:26.592499 1452 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:41229
I20260812 03:08:26.592520 1589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:41229 every 8 connection(s)
I20260812 03:08:26.593604 1452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:08:26.596215 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 1452
I20260812 03:08:26.596386 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance
I20260812 03:08:26.596712 27626 external_mini_cluster.cc:1428] Setting key f9688a38d9317cf091c4c2f13da00aee
Aug 12 03:08:26 dist-test-slave-zpfg krb5kdc[1047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504106, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:26.609992 1138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:26.595747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:47409 (local address 127.26.250.190:39337)
0812 03:08:26.595912 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:26.595917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:26.597040 (+ 1123us) server_negotiation.cc:407] Connection header received
0812 03:08:26.599166 (+ 2126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:26.599173 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:26.599244 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:26.599354 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:26.600577 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:26.601168 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:26.602461 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:26.602702 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:26.602848 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:26.605781 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:26.605786 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:26.605789 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:26.605823 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:26.607665 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:26.608266 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:26.608269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:26.608270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:26.608377 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:26.608770 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:26.608773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:26.608774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:26.608991 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:26.609079 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:26.609664 (+ 585us) server_negotiation.cc:299] Negotiation successful
0812 03:08:26.609835 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 03:08:26.611140 1592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:26.596270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39337 (local address 127.26.250.131:47409)
0812 03:08:26.596848 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:26.596905 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:26.598895 (+ 1990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:26.599508 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:26.599516 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:26.599908 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:26.600425 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:26.600439 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:26.601290 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:26.601292 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:26.602258 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:26.602268 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:26.602901 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:26.602904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:26.603084 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:26.603699 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:26.603717 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:26.605531 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:26.607820 (+ 2289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:26.607826 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:26.607841 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:26.608136 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:26.608501 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:26.608504 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:26.608506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:26.608671 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:26.609097 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:26.609102 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:26.609377 (+ 275us) client_negotiation.cc:769] Sending connection context
0812 03:08:26.609549 (+ 172us) client_negotiation.cc:240] Negotiation successful
0812 03:08:26.609921 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":189,"threads_started":1}
I20260812 03:08:26.612545 1590 heartbeater.cc:347] Connected to a master server at 127.26.250.190:39337
I20260812 03:08:26.612776 1590 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:26.613263 1590 heartbeater.cc:511] Master 127.26.250.190:39337 requested a full tablet report, sending...
I20260812 03:08:26.614459 1093 ts_manager.cc:194] Registered new tserver with Master: 536c614daa9a44498d60d6c76da1a382 (127.26.250.131:41229)
I20260812 03:08:26.615145 1093 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.131@KRBTEST.COM'} at 127.26.250.131:47409
I20260812 03:08:26.624459 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/test-admin.keytab.
Aug 12 03:08:26 dist-test-slave-zpfg krb5kdc[1047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:26.643944 27626 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 03:08:26.647210 1597 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:26.645355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53164 (local address 127.26.250.129:44569)
0812 03:08:26.645877 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:26.645884 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:26.645917 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:08:26.645973 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:26.645980 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:26.646183 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:26.646331 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:26.647010 (+ 679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53164: BlockingRecv error: recv got EOF from 127.0.0.1:53164 (error 108)
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":154,"threads_started":1}
W20260812 03:08:26.647241 27626 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/krb5cc' of type FILE
Aug 12 03:08:26 dist-test-slave-zpfg krb5kdc[1047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:26.648524 27626 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 03:08:26 dist-test-slave-zpfg krb5kdc[1047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:26.649573 27626 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 03:08:26 dist-test-slave-zpfg krb5kdc[1047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.129@KRBTEST.COM
I20260812 03:08:26.660992 1597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:26.650682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53172 (local address 127.26.250.129:44569)
0812 03:08:26.650864 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:26.650869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:26.650888 (+ 19us) server_negotiation.cc:407] Connection header received
0812 03:08:26.651011 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:26.651014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:26.651065 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:26.651342 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:26.651870 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:26.652797 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:26.653707 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:26.654117 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:26.654215 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:26.656809 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:26.656821 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:26.656839 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:26.656871 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:26.658847 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:26.659267 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:26.659272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:26.659274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:26.659405 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:26.659646 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:26.659648 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:26.659650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:26.659914 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:26.660135 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:26.660529 (+ 394us) server_negotiation.cc:299] Negotiation successful
0812 03:08:26.660737 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260812 03:08:26.665292 1597 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:26.664156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53180 (local address 127.26.250.129:44569)
0812 03:08:26.664287 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:26.664290 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:26.664563 (+ 273us) server_negotiation.cc:407] Connection header received
0812 03:08:26.664632 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:26.664634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:26.664673 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:26.664740 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:26.665140 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53180: BlockingRecv error: recv got EOF from 127.0.0.1:53180 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260812 03:08:26.665452 27626 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/krb5cc' of type FILE
Aug 12 03:08:26 dist-test-slave-zpfg krb5kdc[1047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:26.666529 27626 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 03:08:26 dist-test-slave-zpfg krb5kdc[1047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:26.667424 27626 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 03:08:26 dist-test-slave-zpfg krb5kdc[1047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.129@KRBTEST.COM
I20260812 03:08:26.677078 1597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:26.668306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53186 (local address 127.26.250.129:44569)
0812 03:08:26.668454 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:26.668460 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:26.668552 (+ 92us) server_negotiation.cc:407] Connection header received
0812 03:08:26.668714 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:26.668717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:26.668766 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:26.668866 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:26.669382 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:26.669911 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:26.670799 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:26.671049 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:26.671164 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:26.673567 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:26.673571 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:26.673573 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:26.673600 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:26.675301 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:26.675777 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:26.675782 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:26.675784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:26.675872 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:26.676171 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:26.676175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:26.676177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:26.676373 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:26.676517 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:26.676775 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 03:08:26.676899 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260812 03:08:26.680267 1597 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:26.679184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53192 (local address 127.26.250.129:44569)
0812 03:08:26.679345 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:26.679349 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:26.679460 (+ 111us) server_negotiation.cc:407] Connection header received
0812 03:08:26.679596 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:26.679600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:26.679641 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:26.679732 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:26.680156 (+ 424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53192: BlockingRecv error: recv got EOF from 127.0.0.1:53192 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260812 03:08:26.680393 27626 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786504075277362-27626-0/krb5kdc/krb5cc' of type FILE
Aug 12 03:08:26 dist-test-slave-zpfg krb5kdc[1047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:26.681595 27626 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 03:08:26 dist-test-slave-zpfg krb5kdc[1047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:26.682544 27626 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 03:08:26 dist-test-slave-zpfg krb5kdc[1047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.129@KRBTEST.COM
I20260812 03:08:26.691650 1597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:26.683393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53194 (local address 127.26.250.129:44569)
0812 03:08:26.683549 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:26.683552 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:26.683566 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:08:26.683661 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:26.683664 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:26.683731 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:26.683801 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:26.684296 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:26.684831 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:26.685666 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:26.685914 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:26.686096 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:26.688323 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:26.688327 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:26.688329 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:26.688357 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:26.689732 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:26.690142 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:26.690144 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:26.690146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:26.690239 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:26.690789 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:26.690792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:26.690793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:26.690981 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:26.691113 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:26.691333 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 03:08:26.691440 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 03:08:26.692878 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 1162
I20260812 03:08:26.703092 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 1307
I20260812 03:08:26.712947 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 1452
I20260812 03:08:26.720835 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 1062
2026-08-12T03:08:26Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2452 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:08:26 dist-test-slave-zpfg krb5kdc[1611](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:08:26 dist-test-slave-zpfg krb5kdc[1611](info): set up 2 sockets
Aug 12 03:08:26 dist-test-slave-zpfg krb5kdc[1611](info): commencing operation
krb5kdc: starting...
W20260812 03:08:27.832079 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:08:27 dist-test-slave-zpfg krb5kdc[1611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:08:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:08:27Z Disabled control of system clock
WARNING: no policy specified for oryx/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.190.keytab.
Entry for principal oryx/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.190.keytab.
I20260812 03:08:27.974054 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:39831
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.190.keytab
--principal=oryx/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:39227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:39831
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:28.101604 1625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:28.101959 1625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:28.102006 1625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:28.106113 1625 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:08:28.106187 1625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:28.106207 1625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:28.106230 1625 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:08:28.106246 1625 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:08:28.111929 1625 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:39227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:39831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.190.keytab
--principal=oryx/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:39831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:28.113449 1625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:28.115042 1625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:28.125185 1631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:28.125398 1625 server_base.cc:1034] running on GCE node
W20260812 03:08:28.125444 1632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:28.125319 1635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:28.126078 1625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:28.127420 1625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:28.128633 1625 hybrid_clock.cc:648] HybridClock initialized: now 1786504108128615 us; error 34 us; skew 500 ppm
Aug 12 03:08:28 dist-test-slave-zpfg krb5kdc[1611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504108, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:28.145496 1625 init.cc:377] Logged in from keytab as oryx/127.26.250.190@KRBTEST.COM (short username oryx)
I20260812 03:08:28.147051 1625 webserver.cc:460] Webserver started at http://127.26.250.190:45961/ using document root <none> and password file <none>
I20260812 03:08:28.147746 1625 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:28.147855 1625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:28.148110 1625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:28.150552 1625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "c767d9b346ba4b9d9b221992229a332c"
format_stamp: "Formatted at 2026-08-12 03:08:28 on dist-test-slave-zpfg"
server_key: "bee225dc380070683ce9f8dacd47b926"
server_key_iv: "560068a4d5eb3a82cb1ddb5460820e26"
server_key_version: "encryptionkey@0"
I20260812 03:08:28.151276 1625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "c767d9b346ba4b9d9b221992229a332c"
format_stamp: "Formatted at 2026-08-12 03:08:28 on dist-test-slave-zpfg"
server_key: "bee225dc380070683ce9f8dacd47b926"
server_key_iv: "560068a4d5eb3a82cb1ddb5460820e26"
server_key_version: "encryptionkey@0"
I20260812 03:08:28.155745 1625 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 03:08:28.158699 1643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:28.160223 1625 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:08:28.160392 1625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "c767d9b346ba4b9d9b221992229a332c"
format_stamp: "Formatted at 2026-08-12 03:08:28 on dist-test-slave-zpfg"
server_key: "bee225dc380070683ce9f8dacd47b926"
server_key_iv: "560068a4d5eb3a82cb1ddb5460820e26"
server_key_version: "encryptionkey@0"
I20260812 03:08:28.160521 1625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:28.177273 1625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:28.180934 1625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:28.181169 1625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:28.189450 1625 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:39831
I20260812 03:08:28.189859 1698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:39831 every 8 connection(s)
I20260812 03:08:28.190588 1625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:08:28.193297 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 1625
I20260812 03:08:28.193452 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:08:28.193684 1699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:28.193825 27626 external_mini_cluster.cc:1428] Setting key 94c80ff6122a5a4216c3d2f0e76d930c
I20260812 03:08:28.200426 1699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c: Bootstrap starting.
Aug 12 03:08:28 dist-test-slave-zpfg krb5kdc[1611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM
I20260812 03:08:28.203531 1699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:28.204320 1699 log.cc:824] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:28.206907 1699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c: No bootstrap required, opened a new log
I20260812 03:08:28.209961 1702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:28.195483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57384 (local address 127.26.250.190:39831)
0812 03:08:28.195968 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:28.195984 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:28.196022 (+ 38us) server_negotiation.cc:407] Connection header received
0812 03:08:28.196676 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:28.196693 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:28.197032 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:28.197416 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:28.198413 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:28.199486 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:28.200367 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:28.200951 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:28.201134 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:28.204089 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:28.204095 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:28.204109 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:28.204139 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:28.206402 (+ 2263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:28.206879 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:28.206885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:28.206890 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:28.207064 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:28.207418 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:28.207421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:28.207423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:28.207953 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:28.208157 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:28.208488 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 03:08:28.208829 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":139,"threads_started":1}
I20260812 03:08:28.211545 1699 raft_consensus.cc:348] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } }
I20260812 03:08:28.211771 1699 raft_consensus.cc:374] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:28.211833 1699 raft_consensus.cc:729] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c767d9b346ba4b9d9b221992229a332c, State: Initialized, Role: FOLLOWER
I20260812 03:08:28.212352 1699 consensus_queue.cc:260] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } }
I20260812 03:08:28.212540 1699 raft_consensus.cc:388] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:08:28.212630 1699 raft_consensus.cc:482] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:08:28.212731 1699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:28.213739 1699 raft_consensus.cc:504] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } }
I20260812 03:08:28.214097 1699 leader_election.cc:302] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c767d9b346ba4b9d9b221992229a332c; no voters:
I20260812 03:08:28.214534 1699 leader_election.cc:288] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:08:28.214607 1704 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:28.214942 1704 raft_consensus.cc:686] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [term 1 LEADER]: Becoming Leader. State: Replica: c767d9b346ba4b9d9b221992229a332c, State: Running, Role: LEADER
I20260812 03:08:28.215255 1704 consensus_queue.cc:237] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } }
I20260812 03:08:28.215937 1699 sys_catalog.cc:564] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:08:28.217027 1705 sys_catalog.cc:455] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c767d9b346ba4b9d9b221992229a332c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } }
I20260812 03:08:28.217177 1705 sys_catalog.cc:458] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:28.217160 1706 sys_catalog.cc:455] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c767d9b346ba4b9d9b221992229a332c. Latest consensus state: current_term: 1 leader_uuid: "c767d9b346ba4b9d9b221992229a332c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } }
I20260812 03:08:28.217226 1706 sys_catalog.cc:458] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [sys.catalog]: This master's current role is: LEADER
W20260812 03:08:28.220857 1720 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:08:28.220963 1720 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:08:28.221036 1713 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:08:28.221884 1713 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:08:28.227859 1713 catalog_manager.cc:1347] Generated new cluster ID: 762bd694924c4023850f1473ebe1b5bd
I20260812 03:08:28.227949 1713 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:08:28.250605 1713 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:08:28.251670 1713 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:08:28.260284 1713 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c: Generated new TSK 0
I20260812 03:08:28.261102 1713 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.26.250.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.250.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.250.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.129.keytab.
Entry for principal oryx/127.26.250.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.129.keytab.
I20260812 03:08:28.297734 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:0
--local_ip_for_outbound_sockets=127.26.250.129
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39831
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.129.keytab
--principal=oryx/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:39227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:28.421607 1728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:28.421900 1728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:28.421947 1728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:28.426206 1728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:28.426282 1728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:28.426410 1728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:08:28.432014 1728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:39227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.129.keytab
--principal=oryx/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:28.433360 1728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:28.434584 1728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:28.445225 1736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:28.445290 1734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:28.445225 1733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:28.445451 1728 server_base.cc:1034] running on GCE node
I20260812 03:08:28.446049 1728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:28.446831 1728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:28.448067 1728 hybrid_clock.cc:648] HybridClock initialized: now 1786504108448041 us; error 30 us; skew 500 ppm
Aug 12 03:08:28 dist-test-slave-zpfg krb5kdc[1611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504108, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:28.451593 1728 init.cc:377] Logged in from keytab as oryx/127.26.250.129@KRBTEST.COM (short username oryx)
I20260812 03:08:28.452597 1728 webserver.cc:460] Webserver started at http://127.26.250.129:40425/ using document root <none> and password file <none>
I20260812 03:08:28.453236 1728 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:28.453313 1728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:28.453512 1728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:28.456020 1728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/data/instance:
uuid: "f29103f78e19408087681f54d256c8f7"
format_stamp: "Formatted at 2026-08-12 03:08:28 on dist-test-slave-zpfg"
server_key: "16cde3b21c51201318cd6f7e7f89222c"
server_key_iv: "3658d56d08dcc6ff797eb5a78390c770"
server_key_version: "encryptionkey@0"
I20260812 03:08:28.456738 1728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance:
uuid: "f29103f78e19408087681f54d256c8f7"
format_stamp: "Formatted at 2026-08-12 03:08:28 on dist-test-slave-zpfg"
server_key: "16cde3b21c51201318cd6f7e7f89222c"
server_key_iv: "3658d56d08dcc6ff797eb5a78390c770"
server_key_version: "encryptionkey@0"
I20260812 03:08:28.461429 1728 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 03:08:28.464226 1743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:28.465950 1728 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:08:28.466109 1728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "f29103f78e19408087681f54d256c8f7"
format_stamp: "Formatted at 2026-08-12 03:08:28 on dist-test-slave-zpfg"
server_key: "16cde3b21c51201318cd6f7e7f89222c"
server_key_iv: "3658d56d08dcc6ff797eb5a78390c770"
server_key_version: "encryptionkey@0"
I20260812 03:08:28.466261 1728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:28.490130 1728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:28.494319 1728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:28.494633 1728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:28.495352 1728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:28.496310 1728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:28.496389 1728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:28.496469 1728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:28.496515 1728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:28.507054 1728 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:46261
I20260812 03:08:28.507077 1868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:46261 every 8 connection(s)
I20260812 03:08:28.508111 1728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:08:28.516842 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 1728
I20260812 03:08:28.517011 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance
I20260812 03:08:28.517359 27626 external_mini_cluster.cc:1428] Setting key 3ce7c998367b0a3932e7455455a30806
Aug 12 03:08:28 dist-test-slave-zpfg krb5kdc[1611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504108, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM
I20260812 03:08:28.524492 1702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:28.510163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:56067 (local address 127.26.250.190:39831)
0812 03:08:28.510464 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:28.510468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:28.511412 (+ 944us) server_negotiation.cc:407] Connection header received
0812 03:08:28.512929 (+ 1517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:28.512932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:28.512993 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:28.513111 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:28.514455 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:28.515012 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:28.516335 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:28.516697 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:28.516824 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:28.519862 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:28.519870 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:28.519876 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:28.519912 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:28.521952 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:28.522641 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:28.522661 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:28.522662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:28.522751 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:28.523170 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:28.523173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:28.523175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:28.523393 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:28.523515 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:28.524151 (+ 636us) server_negotiation.cc:299] Negotiation successful
0812 03:08:28.524306 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260812 03:08:28.525473 1871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:28.510633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.26.250.129:56067)
0812 03:08:28.511218 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:28.511269 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:28.512691 (+ 1422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:28.513316 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:28.513323 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:28.513715 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:28.514249 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:28.514263 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:28.515166 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:28.515170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:28.516206 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:28.516216 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:28.516995 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:28.517010 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:28.517220 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:28.517959 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:28.517976 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:28.519663 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:28.522102 (+ 2439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:28.522109 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:28.522130 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:28.522518 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:28.522860 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:28.522864 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:28.522866 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:28.523038 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:28.523514 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:28.523522 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:28.523858 (+ 336us) client_negotiation.cc:769] Sending connection context
0812 03:08:28.524087 (+ 229us) client_negotiation.cc:240] Negotiation successful
0812 03:08:28.524492 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":127,"threads_started":1}
I20260812 03:08:28.527101 1869 heartbeater.cc:347] Connected to a master server at 127.26.250.190:39831
I20260812 03:08:28.527403 1869 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:28.527949 1869 heartbeater.cc:511] Master 127.26.250.190:39831 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.26.250.130@KRBTEST.COM; defaulting to no policy
I20260812 03:08:28.529743 1661 ts_manager.cc:194] Registered new tserver with Master: f29103f78e19408087681f54d256c8f7 (127.26.250.129:46261)
I20260812 03:08:28.530992 1661 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.250.129@KRBTEST.COM'} at 127.26.250.129:56067
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.250.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.250.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.130.keytab.
Entry for principal oryx/127.26.250.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.130.keytab.
I20260812 03:08:28.550673 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:0
--local_ip_for_outbound_sockets=127.26.250.130
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39831
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.130.keytab
--principal=oryx/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:39227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:28.671471 1875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:28.671762 1875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:28.671808 1875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:28.675943 1875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:28.676018 1875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:28.676079 1875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:08:28.681634 1875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:39227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.130.keytab
--principal=oryx/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:28.682816 1875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:28.683955 1875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:28.694536 1885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:28.694536 1881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:28.694532 1883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:28.694779 1875 server_base.cc:1034] running on GCE node
I20260812 03:08:28.695274 1875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:28.695922 1875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:28.697142 1875 hybrid_clock.cc:648] HybridClock initialized: now 1786504108697100 us; error 59 us; skew 500 ppm
Aug 12 03:08:28 dist-test-slave-zpfg krb5kdc[1611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504108, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:28.700326 1875 init.cc:377] Logged in from keytab as oryx/127.26.250.130@KRBTEST.COM (short username oryx)
I20260812 03:08:28.701318 1875 webserver.cc:460] Webserver started at http://127.26.250.130:42183/ using document root <none> and password file <none>
I20260812 03:08:28.701931 1875 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:28.702029 1875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:28.702258 1875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:28.704457 1875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/data/instance:
uuid: "e2305e11e279480bb5be780cb9570271"
format_stamp: "Formatted at 2026-08-12 03:08:28 on dist-test-slave-zpfg"
server_key: "67fa6ee90db408889eaff64efb619ca5"
server_key_iv: "5e2fd0f3025d773c7636d82a4934f7fd"
server_key_version: "encryptionkey@0"
I20260812 03:08:28.705073 1875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance:
uuid: "e2305e11e279480bb5be780cb9570271"
format_stamp: "Formatted at 2026-08-12 03:08:28 on dist-test-slave-zpfg"
server_key: "67fa6ee90db408889eaff64efb619ca5"
server_key_iv: "5e2fd0f3025d773c7636d82a4934f7fd"
server_key_version: "encryptionkey@0"
I20260812 03:08:28.709129 1875 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260812 03:08:28.711952 1892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:28.713361 1875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:08:28.713518 1875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "e2305e11e279480bb5be780cb9570271"
format_stamp: "Formatted at 2026-08-12 03:08:28 on dist-test-slave-zpfg"
server_key: "67fa6ee90db408889eaff64efb619ca5"
server_key_iv: "5e2fd0f3025d773c7636d82a4934f7fd"
server_key_version: "encryptionkey@0"
I20260812 03:08:28.713645 1875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:28.740356 1875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:28.743817 1875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:28.744032 1875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:28.744666 1875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:28.745535 1875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:28.745611 1875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:28.745685 1875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:28.745723 1875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:28.755676 1875 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:36363
I20260812 03:08:28.755700 2017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:36363 every 8 connection(s)
I20260812 03:08:28.756654 1875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:08:28.758936 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 1875
I20260812 03:08:28.759080 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance
I20260812 03:08:28.759439 27626 external_mini_cluster.cc:1428] Setting key 4dd044c3279e22a2b485dc64d14bb68f
Aug 12 03:08:28 dist-test-slave-zpfg krb5kdc[1611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504108, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.26.250.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.250.131@KRBTEST.COM" created.
I20260812 03:08:28.774912 1702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:28.758601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:44957 (local address 127.26.250.190:39831)
0812 03:08:28.758788 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:28.758793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:28.759916 (+ 1123us) server_negotiation.cc:407] Connection header received
0812 03:08:28.761948 (+ 2032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:28.761955 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:28.762020 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:28.762092 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:28.763863 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:28.764715 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:28.766112 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:28.766573 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:28.766699 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:28.769846 (+ 3147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:28.769854 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:28.769861 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:28.769900 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:28.771991 (+ 2091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:28.772529 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:28.772531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:28.772533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:28.772609 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:28.773067 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:28.773069 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:28.773070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:28.773310 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:28.773431 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:28.774503 (+ 1072us) server_negotiation.cc:299] Negotiation successful
0812 03:08:28.774668 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 03:08:28.776240 2020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:28.759029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.26.250.130:44957)
0812 03:08:28.759734 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:28.759797 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:28.761684 (+ 1887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:28.762314 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:28.762326 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:28.762937 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:28.763654 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:28.763675 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:28.764929 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:28.764935 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:28.765981 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:28.765991 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:28.766751 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:28.766755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:28.766972 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:28.767732 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:28.767751 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:28.769593 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:28.772119 (+ 2526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:28.772125 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:28.772140 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:28.772434 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:28.772729 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:28.772731 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:28.772733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:28.772951 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:28.773472 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:28.773480 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:28.774021 (+ 541us) client_negotiation.cc:769] Sending connection context
0812 03:08:28.774449 (+ 428us) client_negotiation.cc:240] Negotiation successful
0812 03:08:28.774922 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":180,"threads_started":1}
I20260812 03:08:28.779865 2018 heartbeater.cc:347] Connected to a master server at 127.26.250.190:39831
I20260812 03:08:28.780154 2018 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:28.780845 2018 heartbeater.cc:511] Master 127.26.250.190:39831 requested a full tablet report, sending...
I20260812 03:08:28.782218 1661 ts_manager.cc:194] Registered new tserver with Master: e2305e11e279480bb5be780cb9570271 (127.26.250.130:36363)
I20260812 03:08:28.783099 1661 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.250.130@KRBTEST.COM'} at 127.26.250.130:44957
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.250.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.131.keytab.
Entry for principal oryx/127.26.250.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.131.keytab.
I20260812 03:08:28.798403 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:0
--local_ip_for_outbound_sockets=127.26.250.131
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39831
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.131.keytab
--principal=oryx/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:39227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:28.922850 2024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:28.923151 2024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:28.923199 2024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:28.927526 2024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:28.927599 2024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:28.927662 2024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:08:28.933241 2024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:39227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.131.keytab
--principal=oryx/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:28.934425 2024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:28.935518 2024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:28.946151 2030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:28.946190 2029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:28.946205 2032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:28.946657 2024 server_base.cc:1034] running on GCE node
I20260812 03:08:28.947116 2024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:28.947885 2024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:28.949083 2024 hybrid_clock.cc:648] HybridClock initialized: now 1786504108949065 us; error 41 us; skew 500 ppm
Aug 12 03:08:28 dist-test-slave-zpfg krb5kdc[1611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504108, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:28.956097 2024 init.cc:377] Logged in from keytab as oryx/127.26.250.131@KRBTEST.COM (short username oryx)
I20260812 03:08:28.957170 2024 webserver.cc:460] Webserver started at http://127.26.250.131:37897/ using document root <none> and password file <none>
I20260812 03:08:28.957839 2024 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:28.957942 2024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:28.958176 2024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:28.961663 2024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/data/instance:
uuid: "ac62277b98f34ff58e5d13dee22266bc"
format_stamp: "Formatted at 2026-08-12 03:08:28 on dist-test-slave-zpfg"
server_key: "963c6e3512606289931356108985deb9"
server_key_iv: "35b1b6e793b333815644355ebd4af040"
server_key_version: "encryptionkey@0"
I20260812 03:08:28.962456 2024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance:
uuid: "ac62277b98f34ff58e5d13dee22266bc"
format_stamp: "Formatted at 2026-08-12 03:08:28 on dist-test-slave-zpfg"
server_key: "963c6e3512606289931356108985deb9"
server_key_iv: "35b1b6e793b333815644355ebd4af040"
server_key_version: "encryptionkey@0"
I20260812 03:08:28.968153 2024 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 03:08:28.972157 2041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:28.973811 2024 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:08:28.973963 2024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "ac62277b98f34ff58e5d13dee22266bc"
format_stamp: "Formatted at 2026-08-12 03:08:28 on dist-test-slave-zpfg"
server_key: "963c6e3512606289931356108985deb9"
server_key_iv: "35b1b6e793b333815644355ebd4af040"
server_key_version: "encryptionkey@0"
I20260812 03:08:28.974116 2024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:29.002544 2024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:29.007664 2024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:29.007925 2024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:29.008809 2024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:29.009701 2024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:29.009748 2024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:29.009790 2024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:29.009816 2024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:29.020069 2024 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:39537
I20260812 03:08:29.020153 2165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:39537 every 8 connection(s)
I20260812 03:08:29.021090 2024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:08:29.027411 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 2024
I20260812 03:08:29.027520 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance
I20260812 03:08:29.027820 27626 external_mini_cluster.cc:1428] Setting key bc16441f384a48a3b9397c3aa3aff493
Aug 12 03:08:29 dist-test-slave-zpfg krb5kdc[1611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504108, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM
I20260812 03:08:29.036121 1702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.023306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:52591 (local address 127.26.250.190:39831)
0812 03:08:29.023486 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:29.023490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:29.024400 (+ 910us) server_negotiation.cc:407] Connection header received
0812 03:08:29.025806 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:29.025809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:29.025868 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:29.025945 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:29.027307 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.028026 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.029418 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.029635 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.029716 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.032578 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:29.032581 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:29.032583 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:29.032607 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:29.033974 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.034596 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.034599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.034600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.034676 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.034988 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.034990 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.034991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.035168 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:29.035243 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:29.035870 (+ 627us) server_negotiation.cc:299] Negotiation successful
0812 03:08:29.035991 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 03:08:29.036998 2169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.023710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.26.250.131:52591)
0812 03:08:29.024198 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:29.024244 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:29.025587 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:29.026130 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:29.026137 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:29.026601 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:29.027136 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.027157 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.028173 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.028176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.029286 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.029295 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.029790 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.029794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.030000 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.030730 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:29.030746 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:29.032389 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:29.034085 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.034090 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.034102 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.034506 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.034758 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.034760 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.034761 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.034901 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.035260 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:29.035266 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:29.035579 (+ 313us) client_negotiation.cc:769] Sending connection context
0812 03:08:29.035736 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 03:08:29.036038 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":125,"threads_started":1}
I20260812 03:08:29.038422 2166 heartbeater.cc:347] Connected to a master server at 127.26.250.190:39831
I20260812 03:08:29.038733 2166 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:29.039431 2166 heartbeater.cc:511] Master 127.26.250.190:39831 requested a full tablet report, sending...
I20260812 03:08:29.040793 1661 ts_manager.cc:194] Registered new tserver with Master: ac62277b98f34ff58e5d13dee22266bc (127.26.250.131:39537)
I20260812 03:08:29.041549 1661 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.250.131@KRBTEST.COM'} at 127.26.250.131:52591
I20260812 03:08:29.042273 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.26.250.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.250.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.250.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.189.keytab.
Entry for principal oryx/127.26.250.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.189.keytab.
I20260812 03:08:29.073886 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.189:42549
--webserver_interface=127.26.250.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.189.keytab
--principal=oryx/127.26.250.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:39227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:39831,127.26.250.189:42549
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:29.195838 2172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:29.196183 2172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:29.196298 2172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:29.200687 2172 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:08:29.200783 2172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:29.200821 2172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:29.200881 2172 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:08:29.200937 2172 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:08:29.207387 2172 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:39227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:39831,127.26.250.189:42549
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.189.keytab
--principal=oryx/127.26.250.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.189:42549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.250.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
W20260812 03:08:29.207717 2172 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.26.250.190:39831,127.26.250.189:42549'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 03:08:29.208770 2172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:29.209959 2172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:29.219926 2182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:29.219933 2178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:29.219983 2172 server_base.cc:1034] running on GCE node
W20260812 03:08:29.220315 2180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:29.221045 2172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:29.222276 2172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:29.223573 2172 hybrid_clock.cc:648] HybridClock initialized: now 1786504109223542 us; error 53 us; skew 500 ppm
Aug 12 03:08:29 dist-test-slave-zpfg krb5kdc[1611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504109, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:29.226715 2172 init.cc:377] Logged in from keytab as oryx/127.26.250.189@KRBTEST.COM (short username oryx)
I20260812 03:08:29.227769 2172 webserver.cc:460] Webserver started at http://127.26.250.189:36329/ using document root <none> and password file <none>
I20260812 03:08:29.228415 2172 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:29.228488 2172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:29.228744 2172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:29.231037 2172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-1/data/instance:
uuid: "137f647ad7304220b2b2fe82f9866715"
format_stamp: "Formatted at 2026-08-12 03:08:29 on dist-test-slave-zpfg"
server_key: "34282fb731aee4da1beb8ccc3c6e51fb"
server_key_iv: "61bb95f807513be985e1315c9f4c2ad7"
server_key_version: "encryptionkey@0"
I20260812 03:08:29.231649 2172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-1/wal/instance:
uuid: "137f647ad7304220b2b2fe82f9866715"
format_stamp: "Formatted at 2026-08-12 03:08:29 on dist-test-slave-zpfg"
server_key: "34282fb731aee4da1beb8ccc3c6e51fb"
server_key_iv: "61bb95f807513be985e1315c9f4c2ad7"
server_key_version: "encryptionkey@0"
I20260812 03:08:29.235720 2172 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:08:29.238320 2190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:29.239729 2172 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:08:29.239933 2172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-1/wal
uuid: "137f647ad7304220b2b2fe82f9866715"
format_stamp: "Formatted at 2026-08-12 03:08:29 on dist-test-slave-zpfg"
server_key: "34282fb731aee4da1beb8ccc3c6e51fb"
server_key_iv: "61bb95f807513be985e1315c9f4c2ad7"
server_key_version: "encryptionkey@0"
I20260812 03:08:29.240046 2172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:29.280403 2172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:29.284036 2172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:29.284425 2172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:29.292356 2172 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.189:42549
I20260812 03:08:29.292673 2243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.189:42549 every 8 connection(s)
I20260812 03:08:29.293459 2172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-1/data/info.pb
I20260812 03:08:29.296512 2244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:29.300514 2244 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:08:29.302495 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 2172
Aug 12 03:08:29 dist-test-slave-zpfg krb5kdc[1611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504109, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.189@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM
Aug 12 03:08:29 dist-test-slave-zpfg krb5kdc[1611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.250.189@KRBTEST.COM
I20260812 03:08:29.315423 1702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.302809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57388 (local address 127.26.250.190:39831)
0812 03:08:29.302994 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:29.302999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:29.304123 (+ 1124us) server_negotiation.cc:407] Connection header received
0812 03:08:29.305063 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:29.305066 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:29.305129 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:29.305218 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:29.306416 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.307000 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.308125 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.308438 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.308527 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.311602 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:29.311606 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:29.311608 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:29.311635 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:29.313068 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.313696 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.313700 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.313702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.313788 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.314143 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.314148 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.314150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.314470 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:29.314584 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:29.315096 (+ 512us) server_negotiation.cc:299] Negotiation successful
0812 03:08:29.315227 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260812 03:08:29.316372 2247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.303377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.0.0.1:57388)
0812 03:08:29.303955 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:29.304012 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:29.304822 (+ 810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:29.305300 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:29.305306 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:29.305749 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:29.306079 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.306094 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.307161 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.307168 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.307987 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.308000 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.308608 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.308613 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.308803 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.309534 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:29.309550 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:29.311415 (+ 1865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:29.313194 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.313207 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.313218 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.313584 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.313886 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.313889 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.313891 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.314043 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.314581 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:29.314589 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:29.314870 (+ 281us) client_negotiation.cc:769] Sending connection context
0812 03:08:29.315053 (+ 183us) client_negotiation.cc:240] Negotiation successful
0812 03:08:29.315404 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":65,"thread_start_us":155,"threads_started":1}
I20260812 03:08:29.316376 2248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.303432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59688 (local address 127.26.250.189:42549)
0812 03:08:29.303977 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:29.304001 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:29.304042 (+ 41us) server_negotiation.cc:407] Connection header received
0812 03:08:29.304804 (+ 762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:29.304824 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:29.305033 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:29.305216 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:29.306032 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.307131 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.308027 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.308460 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.308637 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.311876 (+ 3239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:29.311886 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:29.311904 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:29.311933 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:29.313701 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.314165 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.314170 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.314172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.314256 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.314576 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.314579 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.314582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.314854 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:29.315067 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:29.315252 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 03:08:29.315470 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":448,"spinlock_wait_cycles":60672,"thread_start_us":177,"threads_started":1}
I20260812 03:08:29.317859 2244 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:08:29 dist-test-slave-zpfg krb5kdc[1611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504109, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.189@KRBTEST.COM for oryx/127.26.250.189@KRBTEST.COM
I20260812 03:08:29.326508 2247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.318486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:42549 (local address 127.0.0.1:59702)
0812 03:08:29.318610 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:29.318627 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:29.318707 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:29.318943 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:29.318945 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:29.319157 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:29.319283 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.319289 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.319995 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.319998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.320528 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.320535 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.320990 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.320992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.321070 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.321615 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:29.321627 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:29.323164 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:29.325030 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.325033 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.325034 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.325228 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.325471 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.325473 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.325474 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.325569 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.325938 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:29.325940 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:29.326025 (+ 85us) client_negotiation.cc:769] Sending connection context
0812 03:08:29.326104 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 03:08:29.326225 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 03:08:29.326517 2248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.318652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59702 (local address 127.26.250.189:42549)
0812 03:08:29.318757 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:29.318761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:29.318775 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:08:29.318809 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:29.318812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:29.318859 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:29.318937 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:29.319370 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.319897 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.320651 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.320888 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.320972 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.323303 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:29.323306 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:29.323308 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:29.323331 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:29.324911 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.325311 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.325313 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.325314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.325387 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.325646 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.325648 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.325649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.325825 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:29.325936 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:29.326140 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 03:08:29.326281 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":29}
I20260812 03:08:29.330101 2244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715: Bootstrap starting.
I20260812 03:08:29.332726 2244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:29.333514 2244 log.cc:824] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:29.335697 2244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715: No bootstrap required, opened a new log
I20260812 03:08:29.338850 2244 raft_consensus.cc:348] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } }
I20260812 03:08:29.339095 2244 raft_consensus.cc:374] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:29.339180 2244 raft_consensus.cc:729] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 137f647ad7304220b2b2fe82f9866715, State: Initialized, Role: FOLLOWER
I20260812 03:08:29.339622 2244 consensus_queue.cc:260] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } }
I20260812 03:08:29.340157 2251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } } }
I20260812 03:08:29.340299 2251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:08:29.340914 2244 sys_catalog.cc:564] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 03:08:29.346073 2263 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:08:29.346174 2263 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:08:29.348505 2172 master_runner.cc:428] Detected that this master 137f647ad7304220b2b2fe82f9866715 is joining an existing cluster
I20260812 03:08:29.348613 2172 master_runner.cc:432] Initiating AddMaster RPC to add 127.26.250.189:42549
I20260812 03:08:29.359815 2248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.351102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59714 (local address 127.26.250.189:42549)
0812 03:08:29.351215 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:29.351219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:29.351750 (+ 531us) server_negotiation.cc:407] Connection header received
0812 03:08:29.351880 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:29.351886 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:29.351929 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:29.351977 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:29.352552 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.353039 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.354216 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.354610 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.354722 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.356407 (+ 1685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:29.356411 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:29.356413 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:29.356433 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:29.358090 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.358596 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.358600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.358602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.358688 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.358963 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.358967 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.358969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.359159 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:29.359265 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:29.359520 (+ 255us) server_negotiation.cc:299] Negotiation successful
0812 03:08:29.359681 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":42}
I20260812 03:08:29.359833 2272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.351293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:42549 (local address 127.0.0.1:59714)
0812 03:08:29.351660 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:29.351678 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:29.351791 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:29.352108 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:29.352111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:29.352319 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:29.352449 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.352455 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.353162 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.353167 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.354086 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.354095 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.354752 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.354756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.354852 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.355550 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:29.355571 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:29.356303 (+ 732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:29.358204 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.358210 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.358212 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.358501 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.358778 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.358782 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.358783 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.358873 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.359253 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:29.359256 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:29.359346 (+ 90us) client_negotiation.cc:769] Sending connection context
0812 03:08:29.359461 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 03:08:29.359617 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":23,"thread_start_us":113,"threads_started":1}
I20260812 03:08:29.360373 1702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.351078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57398 (local address 127.26.250.190:39831)
0812 03:08:29.351221 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:29.351226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:29.351494 (+ 268us) server_negotiation.cc:407] Connection header received
0812 03:08:29.351668 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:29.351672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:29.351731 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:29.351820 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:29.352402 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.353291 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.354388 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.354629 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.354752 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.356387 (+ 1635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:29.356391 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:29.356394 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:29.356422 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:29.358424 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.358880 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.358883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.358885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.358971 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.359262 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.359265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.359267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.359473 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:29.359542 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:29.359882 (+ 340us) server_negotiation.cc:299] Negotiation successful
0812 03:08:29.360216 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 03:08:29.361088 2271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.351031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.0.0.1:57398)
0812 03:08:29.351401 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:29.351414 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:29.351545 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:29.351834 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:29.351838 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:29.352093 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:29.352270 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.352278 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.353405 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.353408 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.354264 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.354271 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.354873 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.354876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.354967 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.355637 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:29.355652 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:29.356210 (+ 558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:29.358542 (+ 2332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.358546 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.358551 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.358778 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.359076 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.359079 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.359081 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.359162 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.359613 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:29.359615 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:29.359702 (+ 87us) client_negotiation.cc:769] Sending connection context
0812 03:08:29.360721 (+ 1019us) client_negotiation.cc:240] Negotiation successful
0812 03:08:29.360876 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":110,"threads_started":1}
Aug 12 03:08:29 dist-test-slave-zpfg krb5kdc[1611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504108, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.190@KRBTEST.COM for oryx/127.26.250.189@KRBTEST.COM
I20260812 03:08:29.377699 2273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.366012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:42549 (local address 127.0.0.1:59730)
0812 03:08:29.366524 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:29.366582 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:29.366720 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:29.367052 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:29.367058 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:29.367518 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:29.367672 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.367679 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.368506 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.368509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.369554 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.369565 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.370700 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.370703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.370793 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.371605 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:29.371620 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:29.373453 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:29.375710 (+ 2257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.375722 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.375729 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.376071 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.376402 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.376405 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.376407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.376494 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.376975 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:29.376982 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:29.377121 (+ 139us) client_negotiation.cc:769] Sending connection context
0812 03:08:29.377279 (+ 158us) client_negotiation.cc:240] Negotiation successful
0812 03:08:29.377500 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":165,"threads_started":1}
I20260812 03:08:29.377820 2248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.366071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59730 (local address 127.26.250.189:42549)
0812 03:08:29.366241 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:29.366246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:29.366716 (+ 470us) server_negotiation.cc:407] Connection header received
0812 03:08:29.366871 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:29.366873 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:29.366922 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:29.366996 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:29.367805 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.368389 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.369737 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.370458 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.370569 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.373633 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:29.373638 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:29.373640 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:29.373667 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:29.375572 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.376211 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.376215 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.376217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.376301 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.376595 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.376598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.376600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.376818 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:29.376917 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:29.377404 (+ 487us) server_negotiation.cc:299] Negotiation successful
0812 03:08:29.377569 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 03:08:29.380169 1658 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.26.250.189:42549: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "137f647ad7304220b2b2fe82f9866715"
member_type: NON_VOTER
last_known_addr {
host: "127.26.250.189"
port: 42549
}
attrs {
promote: true
}
}
dest_uuid: "c767d9b346ba4b9d9b221992229a332c"
cas_config_opid_index: -1
I20260812 03:08:29.380964 1658 consensus_queue.cc:237] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: NON_VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: true } }
WARNING: no policy specified for oryx/127.26.250.188@KRBTEST.COM; defaulting to no policy
I20260812 03:08:29.383962 2217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:29.384124 2276 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 137f647ad7304220b2b2fe82f9866715 (127.26.250.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: NON_VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: true } } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.250.188@KRBTEST.COM" created.
I20260812 03:08:29.386037 2276 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.26.250.189:42549
I20260812 03:08:29.386401 2217 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [term 1 FOLLOWER]: Refusing update from remote peer c767d9b346ba4b9d9b221992229a332c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 03:08:29.386445 2276 sys_catalog.cc:455] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c767d9b346ba4b9d9b221992229a332c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: NON_VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: true } } }
I20260812 03:08:29.386554 2276 sys_catalog.cc:458] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:29.387280 2276 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [LEADER]: Connected to new peer: Peer: permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: NON_VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:29.388209 2172 master.cc:537] Master@127.26.250.189:42549 shutting down...
W20260812 03:08:29.389217 1647 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260812 03:08:29.390153 1647 consensus_peers.cc:487] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c -> Peer 137f647ad7304220b2b2fe82f9866715 (127.26.250.189:42549): Couldn't send request to peer 137f647ad7304220b2b2fe82f9866715. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 03:08:29.393953 2172 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:29.394389 2172 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:29.394546 2172 tablet_replica.cc:330] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.250.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.188.keytab.
Entry for principal oryx/127.26.250.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.188.keytab.
I20260812 03:08:29.407641 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.188:39479
--webserver_interface=127.26.250.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.188.keytab
--principal=oryx/127.26.250.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:39227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:39831,127.26.250.189:42549,127.26.250.188:39479
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:08:29.410868 2172 master.cc:552] Master@127.26.250.189:42549 shutdown complete.
I20260812 03:08:29.410938 2172 master_runner.cc:305] Clearing existing system tablet
I20260812 03:08:29.411196 2172 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:29.411252 2172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:29.412616 2172 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:08:29.413594 2283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:29.413846 2172 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:29.413971 2172 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:08:29.414004 2172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-1/wal
uuid: "137f647ad7304220b2b2fe82f9866715"
format_stamp: "Formatted at 2026-08-12 03:08:29 on dist-test-slave-zpfg"
server_key: "34282fb731aee4da1beb8ccc3c6e51fb"
server_key_iv: "61bb95f807513be985e1315c9f4c2ad7"
server_key_version: "encryptionkey@0"
I20260812 03:08:29.416455 2172 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:29.419099 2172 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 03:08:29.419196 2172 log.cc:1196] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 03:08:29.419447 2172 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715: Deleting consensus metadata
I20260812 03:08:29.419624 2172 master_runner.cc:315] Copying system tablet from 127.26.250.190:39831
I20260812 03:08:29.420825 2172 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715: tablet copy: Beginning tablet copy session from remote peer at address 127.26.250.190:39831
I20260812 03:08:29.430440 1702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.421560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57400 (local address 127.26.250.190:39831)
0812 03:08:29.421697 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:29.421703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:29.421853 (+ 150us) server_negotiation.cc:407] Connection header received
0812 03:08:29.421980 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:29.421983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:29.422034 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:29.422134 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:29.422705 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.423584 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.424781 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.425143 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.425247 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.426691 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:29.426698 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:29.426703 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:29.426741 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:29.428543 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.429055 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.429058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.429060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.429152 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.429457 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.429463 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.429466 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.429702 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:29.429811 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:29.430118 (+ 307us) server_negotiation.cc:299] Negotiation successful
0812 03:08:29.430248 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 03:08:29.430452 2289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.421440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.0.0.1:57400)
0812 03:08:29.421749 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:29.421765 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:29.421893 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:29.422172 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:29.422175 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:29.422441 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:29.422575 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.422582 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.423718 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.423721 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.424655 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.424666 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.425268 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.425272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.425396 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.426047 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:29.426062 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:29.426531 (+ 469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:29.428668 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.428672 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.428674 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.428948 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.429252 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.429255 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.429257 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.429351 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.429805 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:29.429808 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:29.429924 (+ 116us) client_negotiation.cc:769] Sending connection context
0812 03:08:29.430045 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 03:08:29.430195 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":91,"threads_started":1}
I20260812 03:08:29.431399 1681 tablet_copy_service.cc:140] P c767d9b346ba4b9d9b221992229a332c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 137f647ad7304220b2b2fe82f9866715 ({username='oryx', principal='oryx/127.26.250.189@KRBTEST.COM'} at 127.0.0.1:57400)
I20260812 03:08:29.431603 1681 tablet_copy_service.cc:161] P c767d9b346ba4b9d9b221992229a332c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 137f647ad7304220b2b2fe82f9866715 at {username='oryx', principal='oryx/127.26.250.189@KRBTEST.COM'} at 127.0.0.1:57400: session id = 137f647ad7304220b2b2fe82f9866715-00000000000000000000000000000000
I20260812 03:08:29.433595 1681 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 03:08:29.435420 2172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 03:08:29.437978 2172 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715: tablet copy: Starting download of 0 data blocks...
I20260812 03:08:29.438194 2172 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715: tablet copy: Starting download of 1 WAL segments...
I20260812 03:08:29.440178 2172 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 03:08:29.442409 1681 tablet_copy_service.cc:342] P c767d9b346ba4b9d9b221992229a332c: Request end of tablet copy session 137f647ad7304220b2b2fe82f9866715-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.26.250.189@KRBTEST.COM'} at 127.0.0.1:57400
I20260812 03:08:29.442646 1681 tablet_copy_service.cc:433] P c767d9b346ba4b9d9b221992229a332c: ending tablet copy session 137f647ad7304220b2b2fe82f9866715-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 137f647ad7304220b2b2fe82f9866715
I20260812 03:08:29.450665 2172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:29.452723 2290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:29.452723 2293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:29.453248 2291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:29.454540 2172 server_base.cc:1034] running on GCE node
I20260812 03:08:29.454737 2172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:29.455055 2172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:29.456248 2172 hybrid_clock.cc:648] HybridClock initialized: now 1786504109456233 us; error 36 us; skew 500 ppm
Aug 12 03:08:29 dist-test-slave-zpfg krb5kdc[1611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504109, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:29.458220 2172 init.cc:377] Logged in from keytab as oryx/127.26.250.189@KRBTEST.COM (short username oryx)
I20260812 03:08:29.458750 2172 webserver.cc:460] Webserver started at http://127.26.250.189:35771/ using document root <none> and password file <none>
I20260812 03:08:29.458961 2172 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:29.458995 2172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:29.460393 2172 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:08:29.461520 2301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:29.461813 2172 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 03:08:29.461889 2172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-1/wal
uuid: "137f647ad7304220b2b2fe82f9866715"
format_stamp: "Formatted at 2026-08-12 03:08:29 on dist-test-slave-zpfg"
server_key: "34282fb731aee4da1beb8ccc3c6e51fb"
server_key_iv: "61bb95f807513be985e1315c9f4c2ad7"
server_key_version: "encryptionkey@0"
I20260812 03:08:29.461966 2172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:29.501588 2172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:29.504123 2172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:29.509416 2172 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.189:42549
I20260812 03:08:29.510179 2172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-1/data/info.pb
I20260812 03:08:29.514084 2360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.189:42549 every 8 connection(s)
I20260812 03:08:29.515972 2361 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:08:29.517472 2361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715: Bootstrap starting.
I20260812 03:08:29.529275 2361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:08:29.529633 2361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715: Bootstrap complete.
I20260812 03:08:29.530228 2361 raft_consensus.cc:348] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: NON_VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: true } }
I20260812 03:08:29.530431 2361 raft_consensus.cc:729] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 137f647ad7304220b2b2fe82f9866715, State: Initialized, Role: LEARNER
I20260812 03:08:29.530539 2361 consensus_queue.cc:260] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: NON_VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: true } }
I20260812 03:08:29.531028 2361 sys_catalog.cc:564] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:08:29.531307 2364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: NON_VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: true } } }
I20260812 03:08:29.531541 2364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [sys.catalog]: This master's current role is: LEARNER
I20260812 03:08:29.533658 2375 catalog_manager.cc:1259] Loaded cluster ID: 762bd694924c4023850f1473ebe1b5bd
I20260812 03:08:29.533715 2375 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715: loading cluster ID for follower catalog manager: success
I20260812 03:08:29.533679 1869 heartbeater.cc:503] Master 127.26.250.190:39831 was elected leader, sending a full tablet report...
I20260812 03:08:29.537197 2375 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715: acquiring CA information for follower catalog manager: success
I20260812 03:08:29.538905 2375 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 03:08:29.559474 2281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:29.559831 2281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:29.559886 2281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:29.565201 2281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:08:29.565300 2281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:29.565348 2281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:29.565380 2281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:08:29.565408 2281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:08:29.571969 2281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:39227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:39831,127.26.250.189:42549,127.26.250.188:39479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.188.keytab
--principal=oryx/127.26.250.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.188:39479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.250.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:29.573223 2281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:29.574376 2281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:29.584862 2383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:29.584990 2281 server_base.cc:1034] running on GCE node
W20260812 03:08:29.584865 2381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:29.584865 2380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:29.585997 2281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:29.587520 2281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:29.588713 2281 hybrid_clock.cc:648] HybridClock initialized: now 1786504109588696 us; error 29 us; skew 500 ppm
Aug 12 03:08:29 dist-test-slave-zpfg krb5kdc[1611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504109, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:29.591959 2281 init.cc:377] Logged in from keytab as oryx/127.26.250.188@KRBTEST.COM (short username oryx)
I20260812 03:08:29.593096 2281 webserver.cc:460] Webserver started at http://127.26.250.188:46437/ using document root <none> and password file <none>
I20260812 03:08:29.593771 2281 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:29.593880 2281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:29.594156 2281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:29.596627 2281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-2/data/instance:
uuid: "63499cf6e1c048d6be211a8a4fdaafba"
format_stamp: "Formatted at 2026-08-12 03:08:29 on dist-test-slave-zpfg"
server_key: "a603c3a3f7591dbdc1ca940076aa2c87"
server_key_iv: "8d582cdfaff44cdb4eff0d25b099476f"
server_key_version: "encryptionkey@0"
I20260812 03:08:29.597321 2281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-2/wal/instance:
uuid: "63499cf6e1c048d6be211a8a4fdaafba"
format_stamp: "Formatted at 2026-08-12 03:08:29 on dist-test-slave-zpfg"
server_key: "a603c3a3f7591dbdc1ca940076aa2c87"
server_key_iv: "8d582cdfaff44cdb4eff0d25b099476f"
server_key_version: "encryptionkey@0"
I20260812 03:08:29.601725 2281 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 03:08:29.604620 2390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:29.606103 2281 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:08:29.606438 2281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-2/wal
uuid: "63499cf6e1c048d6be211a8a4fdaafba"
format_stamp: "Formatted at 2026-08-12 03:08:29 on dist-test-slave-zpfg"
server_key: "a603c3a3f7591dbdc1ca940076aa2c87"
server_key_iv: "8d582cdfaff44cdb4eff0d25b099476f"
server_key_version: "encryptionkey@0"
I20260812 03:08:29.606590 2281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:29.631704 2281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:29.634997 2281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:29.635231 2281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:29.643211 2281 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.188:39479
I20260812 03:08:29.643508 2452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.188:39479 every 8 connection(s)
I20260812 03:08:29.644313 2281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-2/data/info.pb
I20260812 03:08:29.646849 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 2281
I20260812 03:08:29.647607 2453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:29.652357 2453 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:08:29 dist-test-slave-zpfg krb5kdc[1611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.250.188@KRBTEST.COM
Aug 12 03:08:29 dist-test-slave-zpfg krb5kdc[1611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504109, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.188@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM
I20260812 03:08:29.664000 2455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.648492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40362 (local address 127.26.250.188:39479)
0812 03:08:29.649049 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:29.649067 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:29.649110 (+ 43us) server_negotiation.cc:407] Connection header received
0812 03:08:29.649971 (+ 861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:29.649990 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:29.650455 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:29.650865 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:29.651811 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.653021 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.654130 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.654865 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.655119 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.658358 (+ 3239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:29.658366 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:29.658398 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:29.658437 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:29.660417 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.660871 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.660879 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.660889 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.661035 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.661422 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.661425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.661426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.662084 (+ 658us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:29.662452 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:29.662704 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 03:08:29.662978 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":171,"threads_started":1}
I20260812 03:08:29.665871 1702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.654443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57402 (local address 127.26.250.190:39831)
0812 03:08:29.654601 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:29.654605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:29.654932 (+ 327us) server_negotiation.cc:407] Connection header received
0812 03:08:29.655469 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:29.655473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:29.655526 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:29.655613 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:29.656745 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.657354 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.658561 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.658878 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.659008 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.661665 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:29.661669 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:29.661672 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:29.661702 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:29.663665 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.664261 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.664264 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.664267 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.664364 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.664772 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.664776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.664778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.664987 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:29.665133 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:29.665535 (+ 402us) server_negotiation.cc:299] Negotiation successful
0812 03:08:29.665686 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 03:08:29.665895 2456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.654269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.0.0.1:57402)
0812 03:08:29.654785 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:29.654839 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:29.654970 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:29.655786 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:29.655794 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:29.656342 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:29.656575 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.656586 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.657505 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.657509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.658442 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.658453 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.659046 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.659054 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.659160 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.659834 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:29.659847 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:29.661507 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:29.663797 (+ 2290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.663813 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.663818 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.664129 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.664494 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.664500 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.664502 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.664600 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.665205 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:29.665210 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:29.665314 (+ 104us) client_negotiation.cc:769] Sending connection context
0812 03:08:29.665472 (+ 158us) client_negotiation.cc:240] Negotiation successful
0812 03:08:29.665677 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"spinlock_wait_cycles":896,"thread_start_us":196,"threads_started":1}
I20260812 03:08:29.667028 2453 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:08:29 dist-test-slave-zpfg krb5kdc[1611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504109, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.188@KRBTEST.COM for oryx/127.26.250.189@KRBTEST.COM
I20260812 03:08:29.676183 2456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.667602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:42549 (local address 127.0.0.1:59742)
0812 03:08:29.667722 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:29.667737 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:29.667807 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:29.668295 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:29.668298 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:29.668491 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:29.668587 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.668592 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.669484 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.669490 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.670022 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.670029 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.670567 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.670570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.670676 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.671147 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:29.671158 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:29.672448 (+ 1290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:29.674409 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.674415 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.674417 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.674652 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.674995 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.675000 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.675002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.675098 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.675544 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:29.675548 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:29.675662 (+ 114us) client_negotiation.cc:769] Sending connection context
0812 03:08:29.675792 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 03:08:29.675956 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 03:08:29.676198 2457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.667772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59742 (local address 127.26.250.189:42549)
0812 03:08:29.668074 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:29.668078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:29.668090 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:08:29.668127 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:29.668129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:29.668174 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:29.668279 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:29.668707 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.669378 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.670138 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.670435 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.670570 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.672594 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:29.672597 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:29.672600 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:29.672634 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:29.674141 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.674780 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.674785 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.674788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.674888 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.675211 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.675215 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.675217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.675436 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:29.675557 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:29.675852 (+ 295us) server_negotiation.cc:299] Negotiation successful
0812 03:08:29.675997 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":109,"threads_started":1}
I20260812 03:08:29.677385 2453 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39479 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:08:29 dist-test-slave-zpfg krb5kdc[1611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504109, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.188@KRBTEST.COM for oryx/127.26.250.188@KRBTEST.COM
I20260812 03:08:29.687871 2456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.677898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:39479 (local address 127.0.0.1:40366)
0812 03:08:29.678050 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:29.678065 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:29.678161 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:29.678483 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:29.678487 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:29.678658 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:29.678754 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.678759 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.680206 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.680209 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.680783 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.680792 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.681531 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.681534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.681602 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.682116 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:29.682127 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:29.684310 (+ 2183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:29.686252 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.686255 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.686256 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.686553 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.686823 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.686825 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.686826 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.686903 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.687411 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:29.687413 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:29.687494 (+ 81us) client_negotiation.cc:769] Sending connection context
0812 03:08:29.687576 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 03:08:29.687683 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260812 03:08:29.687886 2455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.678121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40366 (local address 127.26.250.188:39479)
0812 03:08:29.678257 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:29.678260 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:29.678269 (+ 9us) server_negotiation.cc:407] Connection header received
0812 03:08:29.678293 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:29.678294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:29.678325 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:29.678451 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:29.678908 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.679457 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.680939 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.681402 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.681511 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.684553 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:29.684557 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:29.684559 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:29.684581 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:29.686134 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.686649 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.686653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.686655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.686736 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.687013 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.687016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.687017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.687298 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:29.687387 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:29.687636 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 03:08:29.687743 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:08:29.691129 2453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba: Bootstrap starting.
I20260812 03:08:29.693508 2453 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:29.694264 2453 log.cc:824] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:29.696300 2453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba: No bootstrap required, opened a new log
I20260812 03:08:29.699602 2453 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39479 } }
I20260812 03:08:29.699827 2453 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:29.699898 2453 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63499cf6e1c048d6be211a8a4fdaafba, State: Initialized, Role: FOLLOWER
I20260812 03:08:29.700393 2453 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39479 } }
I20260812 03:08:29.701006 2459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39479 } } }
I20260812 03:08:29.701133 2459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:08:29.701709 2453 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [sys.catalog]: configured and running, proceeding with master startup.
W20260812 03:08:29.707187 2470 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:08:29.707304 2470 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:08:29.710685 2281 master_runner.cc:428] Detected that this master 63499cf6e1c048d6be211a8a4fdaafba is joining an existing cluster
I20260812 03:08:29.710752 2281 master_runner.cc:432] Initiating AddMaster RPC to add 127.26.250.188:39479
I20260812 03:08:29.717778 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 1728
I20260812 03:08:29.723227 2457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.713261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59752 (local address 127.26.250.189:42549)
0812 03:08:29.713378 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:29.713382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:29.714374 (+ 992us) server_negotiation.cc:407] Connection header received
0812 03:08:29.714547 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:29.714550 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:29.714604 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:29.714690 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:29.715230 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.716098 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.717540 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.718041 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.718315 (+ 274us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.719772 (+ 1457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:29.719778 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:29.719786 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:29.719811 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:29.721335 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.721824 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.721829 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.721831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.721920 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.722225 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.722229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.722230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.722507 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:29.722615 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:29.722912 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 03:08:29.723074 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 03:08:29.723242 2476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.713389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:42549 (local address 127.0.0.1:59752)
0812 03:08:29.714250 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:29.714264 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:29.714447 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:29.714706 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:29.714709 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:29.714957 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:29.715092 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.715099 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.716242 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.716245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.717308 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.717318 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.718654 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.718657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.718766 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.719256 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:29.719267 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:29.719587 (+ 320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:29.721460 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.721465 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.721467 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.721709 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.722017 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.722021 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.722023 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.722113 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.722616 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:29.722620 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:29.722732 (+ 112us) client_negotiation.cc:769] Sending connection context
0812 03:08:29.722844 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 03:08:29.723016 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":219,"threads_started":1}
I20260812 03:08:29.730571 2455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.713592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40380 (local address 127.26.250.188:39479)
0812 03:08:29.713687 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:29.713691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:29.723516 (+ 9825us) server_negotiation.cc:407] Connection header received
0812 03:08:29.723665 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:29.723669 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:29.723712 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:29.723760 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:29.724340 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.724952 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.725664 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.725888 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.726001 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.727263 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:29.727266 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:29.727269 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:29.727292 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:29.728819 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.729254 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.729260 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.729262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.729341 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.729615 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.729621 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.729623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.729849 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:29.729959 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:29.730223 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 03:08:29.730412 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":93,"server-negotiator.queue_time_us":29}
I20260812 03:08:29.730571 2476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.713556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:39479 (local address 127.0.0.1:40380)
0812 03:08:29.723442 (+ 9886us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:29.723459 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:29.723576 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:29.723882 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:29.723887 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:29.724062 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:29.724212 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.724222 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.725049 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.725051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.725547 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.725552 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.726009 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.726013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.726152 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.726731 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:29.726751 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:29.727106 (+ 355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:29.728949 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.728952 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.728954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.729156 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.729440 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.729442 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.729444 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.729526 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.729948 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:29.729951 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:29.730047 (+ 96us) client_negotiation.cc:769] Sending connection context
0812 03:08:29.730181 (+ 134us) client_negotiation.cc:240] Negotiation successful
0812 03:08:29.730361 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9814,"mutex_wait_us":19,"thread_start_us":90,"threads_started":1}
I20260812 03:08:29.735931 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:46261
--local_ip_for_outbound_sockets=127.26.250.129
--tserver_master_addrs=127.26.250.190:39831
--webserver_port=40425
--webserver_interface=127.26.250.129
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.129.keytab
--principal=oryx/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:39227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.26.250.190:39831,127.26.250.189:42549
--tserver_master_addrs=127.26.250.190:39831,127.26.250.189:42549,127.26.250.188:39479 with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:08:29.740026 2459 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:08:29.740334 2459 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39479 } }
I20260812 03:08:29.741427 2459 leader_election.cc:288] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c767d9b346ba4b9d9b221992229a332c (127.26.250.190:39831), 137f647ad7304220b2b2fe82f9866715 (127.26.250.189:42549)
I20260812 03:08:29.742038 1671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63499cf6e1c048d6be211a8a4fdaafba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c767d9b346ba4b9d9b221992229a332c" is_pre_election: true
I20260812 03:08:29.742038 2332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63499cf6e1c048d6be211a8a4fdaafba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "137f647ad7304220b2b2fe82f9866715" is_pre_election: true
I20260812 03:08:29.742472 2332 raft_consensus.cc:1799] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [term 1 LEARNER]: Handling vote request from an unknown peer 63499cf6e1c048d6be211a8a4fdaafba
I20260812 03:08:29.742635 2332 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 63499cf6e1c048d6be211a8a4fdaafba for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260812 03:08:29.743054 2395 leader_election.cc:302] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 63499cf6e1c048d6be211a8a4fdaafba; no voters: 137f647ad7304220b2b2fe82f9866715, c767d9b346ba4b9d9b221992229a332c
I20260812 03:08:29.743690 2459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:29.745316 2459 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 03:08:29.747586 1702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.713110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57412 (local address 127.26.250.190:39831)
0812 03:08:29.713252 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:29.713257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:29.713937 (+ 680us) server_negotiation.cc:407] Connection header received
0812 03:08:29.714063 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:29.714070 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:29.714126 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:29.714212 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:29.714843 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.715517 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.716767 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.717163 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.717246 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.743514 (+ 26268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:29.743522 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:29.743527 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:29.743563 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:29.745551 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.746093 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.746100 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.746103 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.746197 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.746566 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.746570 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.746572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.746786 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:29.747002 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:29.747318 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 03:08:29.747445 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 03:08:29.747617 2477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.713048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.0.0.1:57412)
0812 03:08:29.713827 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:29.713845 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:29.713973 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:29.714274 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:29.714277 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:29.714551 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:29.714711 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.714720 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.715628 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.715634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.716625 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.716635 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.717411 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.717417 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.717561 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.718540 (+ 979us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:29.718600 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:29.743330 (+ 24730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:29.745701 (+ 2371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.745704 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.745705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.745978 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.746303 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.746307 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.746309 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.746465 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.746989 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:29.746992 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:29.747112 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 03:08:29.747227 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 03:08:29.747401 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"thread_start_us":129,"threads_started":1}
Aug 12 03:08:29 dist-test-slave-zpfg krb5kdc[1611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504108, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.190@KRBTEST.COM for oryx/127.26.250.188@KRBTEST.COM
I20260812 03:08:29.760942 2273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.751888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:39479 (local address 127.0.0.1:40388)
0812 03:08:29.752025 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:29.752040 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:29.752128 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:29.752569 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:29.752572 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:29.752789 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:29.752887 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.752893 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.753616 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.753621 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.754182 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.754190 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.754723 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.754727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.754828 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.755380 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:29.755394 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:29.757051 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:29.759301 (+ 2250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.759304 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.759306 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.759563 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.759864 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.759867 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.759869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.759975 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.760445 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:29.760448 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:29.760545 (+ 97us) client_negotiation.cc:769] Sending connection context
0812 03:08:29.760625 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 03:08:29.760782 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 03:08:29.761552 2455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.751953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40388 (local address 127.26.250.188:39479)
0812 03:08:29.752071 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:29.752078 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:29.752094 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:08:29.752400 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:29.752403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:29.752465 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:29.752551 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:29.753005 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.753507 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.754302 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.754587 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.754716 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.757244 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:29.757249 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:29.757251 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:29.757279 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:29.759158 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.759688 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.759691 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.759693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.759766 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.760093 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.760096 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.760098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.760334 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:29.761151 (+ 817us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:29.761241 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 03:08:29.761362 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:08:29.762651 1658 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.26.250.188:39479: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba"
member_type: NON_VOTER
last_known_addr {
host: "127.26.250.188"
port: 39479
}
attrs {
promote: true
}
}
dest_uuid: "c767d9b346ba4b9d9b221992229a332c"
cas_config_opid_index: 5
I20260812 03:08:29.763314 1658 consensus_queue.cc:237] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: NON_VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: true } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: NON_VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: true } }
I20260812 03:08:29.764474 2276 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 63499cf6e1c048d6be211a8a4fdaafba (127.26.250.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: NON_VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: true } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: NON_VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: true } } }
I20260812 03:08:29.765846 2421 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 1 FOLLOWER]: Refusing update from remote peer c767d9b346ba4b9d9b221992229a332c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 03:08:29.766155 2276 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.26.250.188:39479
I20260812 03:08:29.766394 2276 sys_catalog.cc:455] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c767d9b346ba4b9d9b221992229a332c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: NON_VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: true } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: NON_VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: true } } }
I20260812 03:08:29.766515 2276 sys_catalog.cc:458] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:29.766633 2276 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [LEADER]: Connected to new peer: Peer: permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: NON_VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:29.771317 2421 raft_consensus.cc:858] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39479 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: NON_VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: NON_VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: true } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: NON_VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: true } }
W20260812 03:08:29.771471 2421 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39479 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: NON_VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: NON_VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: true } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: NON_VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: true } }
I20260812 03:08:29.772289 2273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.764692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:42549 (local address 127.0.0.1:59764)
0812 03:08:29.766867 (+ 2175us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:29.766883 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:29.766970 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:29.767540 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:29.767544 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:29.767566 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:29.767674 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.767683 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.768841 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.768845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.770034 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.770046 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.771713 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.771717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.771814 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.771869 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 03:08:29.771944 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 03:08:29.772032 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2091}
I20260812 03:08:29.773083 2457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.764981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59764 (local address 127.26.250.189:42549)
0812 03:08:29.767151 (+ 2170us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:29.767156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:29.767171 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:08:29.767222 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:29.767225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:29.767291 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:29.767390 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:29.767813 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.768552 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.770284 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.771177 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.771299 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.772819 (+ 1520us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:29.772860 (+ 41us) server_negotiation.cc:299] Negotiation successful
0812 03:08:29.772947 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2071}
I20260812 03:08:29.773154 2421 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 137f647ad7304220b2b2fe82f9866715 (127.26.250.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: NON_VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: true } } }
I20260812 03:08:29.773911 2332 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [term 1 LEARNER]: Refusing update from remote peer c767d9b346ba4b9d9b221992229a332c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 03:08:29.774447 2277 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [LEADER]: Connected to new peer: Peer: permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: NON_VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 03:08:29.775334 2483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c767d9b346ba4b9d9b221992229a332c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: NON_VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: true } } }
I20260812 03:08:29.775447 2483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260812 03:08:29.775535 2483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c767d9b346ba4b9d9b221992229a332c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: NON_VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: true } } }
I20260812 03:08:29.775596 2483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260812 03:08:29.776502 2332 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 63499cf6e1c048d6be211a8a4fdaafba (127.26.250.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: NON_VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: true } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: NON_VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: true } } }
I20260812 03:08:29.778841 2364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c767d9b346ba4b9d9b221992229a332c. Latest consensus state: current_term: 1 leader_uuid: "c767d9b346ba4b9d9b221992229a332c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: NON_VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: true } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: NON_VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: true } } }
I20260812 03:08:29.778944 2277 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c: attempting to promote NON_VOTER 137f647ad7304220b2b2fe82f9866715 to VOTER
I20260812 03:08:29.778960 2364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [sys.catalog]: This master's current role is: LEARNER
I20260812 03:08:29.779055 2364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c767d9b346ba4b9d9b221992229a332c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: NON_VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: true } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: NON_VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: true } } }
I20260812 03:08:29.779122 2364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [sys.catalog]: This master's current role is: LEARNER
I20260812 03:08:29.779418 2277 consensus_queue.cc:237] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: NON_VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: true } }
I20260812 03:08:29.781370 2332 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [term 1 LEARNER]: Refusing update from remote peer c767d9b346ba4b9d9b221992229a332c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 03:08:29.781978 2459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [sys.catalog]: SysCatalogTable state changed. Reason: New leader c767d9b346ba4b9d9b221992229a332c. Latest consensus state: current_term: 1 leader_uuid: "c767d9b346ba4b9d9b221992229a332c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: NON_VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: true } } }
I20260812 03:08:29.782075 2459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260812 03:08:29.782383 2486 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [LEADER]: Connected to new peer: Peer: permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 03:08:29.782989 2281 master.cc:537] Master@127.26.250.188:39479 shutting down...
I20260812 03:08:29.786072 2277 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 137f647ad7304220b2b2fe82f9866715 (127.26.250.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: NON_VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: true } } }
I20260812 03:08:29.788758 2277 sys_catalog.cc:455] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c767d9b346ba4b9d9b221992229a332c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: NON_VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: true } } }
I20260812 03:08:29.789086 2277 sys_catalog.cc:458] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:29.789491 2277 sys_catalog.cc:455] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c767d9b346ba4b9d9b221992229a332c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: NON_VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: true } } }
I20260812 03:08:29.789578 2277 sys_catalog.cc:458] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:29.790537 2420 raft_consensus.cc:1467] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 1 NON_PARTICIPANT]: Allowing update even though not a member of the config
I20260812 03:08:29.790681 2420 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 1 NON_PARTICIPANT]: Refusing update from remote peer c767d9b346ba4b9d9b221992229a332c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 03:08:29.790534 2332 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 137f647ad7304220b2b2fe82f9866715 (127.26.250.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: NON_VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: true } } }
I20260812 03:08:29.791602 2276 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [LEADER]: Connected to new peer: Peer: permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: NON_VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 03:08:29.790091 2018 heartbeater.cc:503] Master 127.26.250.190:39831 was elected leader, sending a full tablet report...
W20260812 03:08:29.794806 1645 consensus_peers.cc:487] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c -> Peer 63499cf6e1c048d6be211a8a4fdaafba (127.26.250.188:39479): Couldn't send request to peer 63499cf6e1c048d6be211a8a4fdaafba. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 03:08:29.795014 2485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c767d9b346ba4b9d9b221992229a332c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: NON_VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: true } } }
I20260812 03:08:29.795157 2485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:08:29.796540 2281 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260812 03:08:29.796821 2281 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260812 03:08:29.796937 2281 tablet_replica.cc:330] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba: stopping tablet replica
I20260812 03:08:29.804296 2281 master.cc:552] Master@127.26.250.188:39479 shutdown complete.
I20260812 03:08:29.804387 2281 master_runner.cc:305] Clearing existing system tablet
I20260812 03:08:29.804623 2281 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:29.804705 2281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:29.805800 2281 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 03:08:29.806740 2492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:29.806967 2281 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:29.807101 2281 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:08:29.807144 2281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-2/wal
uuid: "63499cf6e1c048d6be211a8a4fdaafba"
format_stamp: "Formatted at 2026-08-12 03:08:29 on dist-test-slave-zpfg"
server_key: "a603c3a3f7591dbdc1ca940076aa2c87"
server_key_iv: "8d582cdfaff44cdb4eff0d25b099476f"
server_key_version: "encryptionkey@0"
I20260812 03:08:29.808614 2281 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:29.810853 2281 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 03:08:29.810958 2281 log.cc:1196] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 03:08:29.811234 2281 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba: Deleting consensus metadata
I20260812 03:08:29.811431 2281 master_runner.cc:315] Copying system tablet from 127.26.250.190:39831
I20260812 03:08:29.812652 2281 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba: tablet copy: Beginning tablet copy session from remote peer at address 127.26.250.190:39831
I20260812 03:08:29.825037 2498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.816894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.0.0.1:57428)
0812 03:08:29.817169 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:29.817185 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:29.817439 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:29.817842 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:29.817846 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:29.818127 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:29.818296 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.818302 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.819277 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.819281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.819881 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.819890 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.820374 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.820378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.820476 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.821062 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:29.821083 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:29.821425 (+ 342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:29.823296 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.823301 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.823303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.823644 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.823944 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.823948 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.823949 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.824039 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.824437 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:29.824440 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:29.824553 (+ 113us) client_negotiation.cc:769] Sending connection context
0812 03:08:29.824665 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 03:08:29.824829 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":122,"threads_started":1}
I20260812 03:08:29.825026 1702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.816682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57428 (local address 127.26.250.190:39831)
0812 03:08:29.816825 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:29.816831 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:29.817350 (+ 519us) server_negotiation.cc:407] Connection header received
0812 03:08:29.817620 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:29.817624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:29.817683 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:29.817771 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:29.818490 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.819108 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.820006 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.820247 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.820360 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.821572 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:29.821576 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:29.821579 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:29.821605 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:29.823175 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.823752 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.823755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.823757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.823848 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.824130 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.824135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.824137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.824339 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:29.824448 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:29.824721 (+ 273us) server_negotiation.cc:299] Negotiation successful
0812 03:08:29.824851 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:08:29.825904 1681 tablet_copy_service.cc:140] P c767d9b346ba4b9d9b221992229a332c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 63499cf6e1c048d6be211a8a4fdaafba ({username='oryx', principal='oryx/127.26.250.188@KRBTEST.COM'} at 127.0.0.1:57428)
I20260812 03:08:29.826026 1681 tablet_copy_service.cc:161] P c767d9b346ba4b9d9b221992229a332c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 63499cf6e1c048d6be211a8a4fdaafba at {username='oryx', principal='oryx/127.26.250.188@KRBTEST.COM'} at 127.0.0.1:57428: session id = 63499cf6e1c048d6be211a8a4fdaafba-00000000000000000000000000000000
I20260812 03:08:29.827054 1681 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 03:08:29.828261 2281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 03:08:29.830206 2281 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba: tablet copy: Starting download of 0 data blocks...
I20260812 03:08:29.830482 2281 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba: tablet copy: Starting download of 1 WAL segments...
I20260812 03:08:29.832085 2281 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 03:08:29.833907 1681 tablet_copy_service.cc:342] P c767d9b346ba4b9d9b221992229a332c: Request end of tablet copy session 63499cf6e1c048d6be211a8a4fdaafba-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.26.250.188@KRBTEST.COM'} at 127.0.0.1:57428
I20260812 03:08:29.834017 1681 tablet_copy_service.cc:433] P c767d9b346ba4b9d9b221992229a332c: ending tablet copy session 63499cf6e1c048d6be211a8a4fdaafba-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 63499cf6e1c048d6be211a8a4fdaafba
I20260812 03:08:29.840605 2281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:29.842478 2503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:29.842517 2501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:29.842845 2500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:29.843083 2281 server_base.cc:1034] running on GCE node
I20260812 03:08:29.843245 2281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:29.843515 2281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:29.844647 2281 hybrid_clock.cc:648] HybridClock initialized: now 1786504109844635 us; error 31 us; skew 500 ppm
Aug 12 03:08:29 dist-test-slave-zpfg krb5kdc[1611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504109, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:29.846153 2281 init.cc:377] Logged in from keytab as oryx/127.26.250.188@KRBTEST.COM (short username oryx)
I20260812 03:08:29.846719 2281 webserver.cc:460] Webserver started at http://127.26.250.188:36053/ using document root <none> and password file <none>
I20260812 03:08:29.846961 2281 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:29.847024 2281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:29.848179 2281 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:08:29.849134 2510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:29.849370 2281 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 03:08:29.849468 2281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-2/wal
uuid: "63499cf6e1c048d6be211a8a4fdaafba"
format_stamp: "Formatted at 2026-08-12 03:08:29 on dist-test-slave-zpfg"
server_key: "a603c3a3f7591dbdc1ca940076aa2c87"
server_key_iv: "8d582cdfaff44cdb4eff0d25b099476f"
server_key_version: "encryptionkey@0"
I20260812 03:08:29.849556 2281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:29.856040 2281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:29.858219 2281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:29.863453 2281 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.188:39479
I20260812 03:08:29.864324 2566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.188:39479 every 8 connection(s)
I20260812 03:08:29.864367 2281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/master-2/data/info.pb
I20260812 03:08:29.866365 2567 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:08:29.867503 2567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba: Bootstrap starting.
I20260812 03:08:29.876453 2567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:08:29.876820 2567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba: Bootstrap complete.
I20260812 03:08:29.877487 2567 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: NON_VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: true } }
I20260812 03:08:29.877673 2567 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 63499cf6e1c048d6be211a8a4fdaafba, State: Initialized, Role: LEARNER
I20260812 03:08:29.877800 2567 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: NON_VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: true } }
I20260812 03:08:29.878253 2570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: NON_VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: true } } }
I20260812 03:08:29.878412 2567 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:08:29.878535 2570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [sys.catalog]: This master's current role is: LEARNER
I20260812 03:08:29.882110 2581 catalog_manager.cc:1259] Loaded cluster ID: 762bd694924c4023850f1473ebe1b5bd
I20260812 03:08:29.882259 2581 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba: loading cluster ID for follower catalog manager: success
I20260812 03:08:29.884637 2581 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba: acquiring CA information for follower catalog manager: success
I20260812 03:08:29.886164 2581 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 03:08:29.899367 2479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:29.899655 2479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:29.899701 2479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:29.904035 2479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:29.904122 2479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:29.904191 2479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:08:29.910015 2479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:39227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.129.keytab
--principal=oryx/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:46261
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=40425
--tserver_master_addrs=127.26.250.190:39831,127.26.250.189:42549,127.26.250.188:39479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:29.911300 2479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:29.912528 2479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:29.923197 2591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:29.923214 2586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:29.923418 2588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:29.924126 2479 server_base.cc:1034] running on GCE node
I20260812 03:08:29.924613 2479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:29.925424 2479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:29.926661 2479 hybrid_clock.cc:648] HybridClock initialized: now 1786504109926630 us; error 44 us; skew 500 ppm
Aug 12 03:08:29 dist-test-slave-zpfg krb5kdc[1611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504109, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:29.929831 2479 init.cc:377] Logged in from keytab as oryx/127.26.250.129@KRBTEST.COM (short username oryx)
I20260812 03:08:29.931149 2479 webserver.cc:460] Webserver started at http://127.26.250.129:40425/ using document root <none> and password file <none>
I20260812 03:08:29.931843 2479 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:29.931929 2479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:29.936601 2479 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:08:29.938926 2598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:29.940358 2479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:08:29.940469 2479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "f29103f78e19408087681f54d256c8f7"
format_stamp: "Formatted at 2026-08-12 03:08:28 on dist-test-slave-zpfg"
server_key: "16cde3b21c51201318cd6f7e7f89222c"
server_key_iv: "3658d56d08dcc6ff797eb5a78390c770"
server_key_version: "encryptionkey@0"
I20260812 03:08:29.941027 2479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:29.957937 2479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:29.961196 2479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:29.961437 2479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:29.962087 2479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:29.963045 2479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:29.963094 2479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:29.963172 2479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:29.963204 2479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:29.972675 2479 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:46261
I20260812 03:08:29.972746 2721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:46261 every 8 connection(s)
I20260812 03:08:29.973800 2479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:08:29.976282 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 2479
I20260812 03:08:29.976496 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 1875
I20260812 03:08:29.988961 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:36363
--local_ip_for_outbound_sockets=127.26.250.130
--tserver_master_addrs=127.26.250.190:39831
--webserver_port=42183
--webserver_interface=127.26.250.130
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.130.keytab
--principal=oryx/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:39227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.26.250.190:39831,127.26.250.189:42549
--tserver_master_addrs=127.26.250.190:39831,127.26.250.189:42549,127.26.250.188:39479 with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:08:29 dist-test-slave-zpfg krb5kdc[1611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504109, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.189@KRBTEST.COM
Aug 12 03:08:29 dist-test-slave-zpfg krb5kdc[1611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504109, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM
Aug 12 03:08:29 dist-test-slave-zpfg krb5kdc[1611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504109, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.188@KRBTEST.COM
I20260812 03:08:29.997738 1702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.976436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:44555 (local address 127.26.250.190:39831)
0812 03:08:29.976728 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:29.976734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:29.980376 (+ 3642us) server_negotiation.cc:407] Connection header received
0812 03:08:29.981080 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:29.981083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:29.981151 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:29.981229 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:29.983447 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.984283 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.986591 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.986949 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.987038 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.991676 (+ 4638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:29.991681 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:29.991684 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:29.991719 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:29.994437 (+ 2718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.995354 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.995359 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.995361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.995464 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.996022 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.996026 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.996029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.996317 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:29.996418 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:29.997375 (+ 957us) server_negotiation.cc:299] Negotiation successful
0812 03:08:29.997535 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199}
I20260812 03:08:29.997745 2457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.976262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:47657 (local address 127.26.250.189:42549)
0812 03:08:29.976607 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:29.976614 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:29.978989 (+ 2375us) server_negotiation.cc:407] Connection header received
0812 03:08:29.981171 (+ 2182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:29.981179 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:29.981268 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:29.981334 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:29.983449 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.984433 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.986599 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.987083 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.987209 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.991676 (+ 4467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:29.991681 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:29.991684 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:29.991733 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:29.994618 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.995355 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.995359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.995361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.995464 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.996016 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.996020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.996021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.996317 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:29.996418 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:29.997375 (+ 957us) server_negotiation.cc:299] Negotiation successful
0812 03:08:29.997535 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230}
I20260812 03:08:29.997833 2730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.978485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:52023 (local address 127.26.250.188:39479)
0812 03:08:29.979065 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:29.979102 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:29.980619 (+ 1517us) server_negotiation.cc:407] Connection header received
0812 03:08:29.981168 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:29.981172 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:29.981230 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:29.981307 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:29.983438 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.984525 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.986589 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.986959 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.987048 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.992814 (+ 5766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:29.992820 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:29.992826 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:29.992862 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:29.995020 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.995611 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.995617 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.995620 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.995723 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:29.996078 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:29.996084 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:29.996088 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:29.996415 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:29.996520 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:29.997394 (+ 874us) server_negotiation.cc:299] Negotiation successful
0812 03:08:29.997570 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":193,"threads_started":1}
I20260812 03:08:29.999987 2728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.976709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.26.250.129:44555)
0812 03:08:29.980250 (+ 3541us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:29.980269 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:29.980816 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:29.981647 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:29.981657 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:29.982398 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:29.983257 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.983278 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.984477 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.984480 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.986438 (+ 1958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.986449 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.987237 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.987240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.987567 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.988581 (+ 1014us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:29.988602 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:29.991417 (+ 2815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:29.994613 (+ 3196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.994621 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.994643 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.995209 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.995594 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.995597 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.995600 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.995881 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.996506 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:29.996515 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:29.997020 (+ 505us) client_negotiation.cc:769] Sending connection context
0812 03:08:29.998591 (+ 1571us) client_negotiation.cc:240] Negotiation successful
0812 03:08:29.998765 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3417,"mutex_wait_us":683,"spinlock_wait_cycles":79872,"thread_start_us":202,"threads_started":1}
I20260812 03:08:30.000001 2726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.978134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:39479 (local address 127.26.250.129:52023)
0812 03:08:29.980521 (+ 2387us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:29.980534 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:29.980816 (+ 282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:29.981647 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:29.981656 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:29.982371 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:29.983257 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.983278 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.984645 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.984649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.986438 (+ 1789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.986449 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.987237 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.987240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.987570 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.989056 (+ 1486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:29.990088 (+ 1032us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:29.992610 (+ 2522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:29.995170 (+ 2560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.995176 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.995179 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.995472 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.995849 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.995854 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.995858 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.995955 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.996592 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:29.996597 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:29.997020 (+ 423us) client_negotiation.cc:769] Sending connection context
0812 03:08:29.998160 (+ 1140us) client_negotiation.cc:240] Negotiation successful
0812 03:08:29.998696 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2327,"mutex_wait_us":845,"spinlock_wait_cycles":47104,"thread_start_us":190,"threads_started":1}
I20260812 03:08:30.000006 2729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:29.976709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:42549 (local address 127.26.250.129:47657)
0812 03:08:29.978787 (+ 2078us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:29.978857 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:29.980811 (+ 1954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:29.981648 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:29.981658 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:29.982404 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:29.983259 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.983281 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.984604 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.984611 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.986425 (+ 1814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:29.986442 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:29.987463 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:29.987470 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:29.987718 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:29.988847 (+ 1129us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:29.988860 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:29.991417 (+ 2557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:29.994764 (+ 3347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.994768 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.994770 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.995209 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.995594 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:29.995597 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:29.995600 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:29.995881 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:29.996506 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:29.996515 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:29.997020 (+ 505us) client_negotiation.cc:769] Sending connection context
0812 03:08:29.998035 (+ 1015us) client_negotiation.cc:240] Negotiation successful
0812 03:08:29.998573 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1735,"mutex_wait_us":356,"spinlock_wait_cycles":38016,"thread_start_us":912,"threads_started":1}
I20260812 03:08:30.001600 2723 heartbeater.cc:347] Connected to a master server at 127.26.250.190:39831
I20260812 03:08:30.001708 2722 heartbeater.cc:347] Connected to a master server at 127.26.250.188:39479
I20260812 03:08:30.001781 2724 heartbeater.cc:347] Connected to a master server at 127.26.250.189:42549
I20260812 03:08:30.001888 2723 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:30.001892 2724 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:30.001888 2722 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:30.002678 2723 heartbeater.cc:511] Master 127.26.250.190:39831 requested a full tablet report, sending...
I20260812 03:08:30.002689 2722 heartbeater.cc:511] Master 127.26.250.188:39479 requested a full tablet report, sending...
I20260812 03:08:30.002753 2724 heartbeater.cc:511] Master 127.26.250.189:42549 requested a full tablet report, sending...
I20260812 03:08:30.004354 1658 ts_manager.cc:194] Re-registered known tserver with Master: f29103f78e19408087681f54d256c8f7 (127.26.250.129:46261)
I20260812 03:08:30.004627 2321 ts_manager.cc:194] Registered new tserver with Master: f29103f78e19408087681f54d256c8f7 (127.26.250.129:46261)
I20260812 03:08:30.004626 2529 ts_manager.cc:194] Registered new tserver with Master: f29103f78e19408087681f54d256c8f7 (127.26.250.129:46261)
I20260812 03:08:30.005414 1658 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.250.129@KRBTEST.COM'} at 127.26.250.129:44555
I20260812 03:08:30.044310 2166 heartbeater.cc:503] Master 127.26.250.190:39831 was elected leader, sending a full tablet report...
W20260812 03:08:30.118925 2731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:30.119233 2731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:30.119280 2731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:30.124181 2731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:30.124285 2731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:30.124408 2731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:08:30.130234 2731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:39227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.130.keytab
--principal=oryx/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:36363
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=42183
--tserver_master_addrs=127.26.250.190:39831,127.26.250.189:42549,127.26.250.188:39479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:30.131578 2731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:30.132719 2731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:30.143095 2740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:30.143090 2737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:30.143087 2736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:30.143260 2731 server_base.cc:1034] running on GCE node
I20260812 03:08:30.143976 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:30.144737 2731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:30.145992 2731 hybrid_clock.cc:648] HybridClock initialized: now 1786504110145974 us; error 40 us; skew 500 ppm
Aug 12 03:08:30 dist-test-slave-zpfg krb5kdc[1611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504110, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:30.149111 2731 init.cc:377] Logged in from keytab as oryx/127.26.250.130@KRBTEST.COM (short username oryx)
I20260812 03:08:30.150228 2731 webserver.cc:460] Webserver started at http://127.26.250.130:42183/ using document root <none> and password file <none>
I20260812 03:08:30.150866 2731 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:30.150981 2731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:30.155202 2731 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:08:30.157619 2747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:30.159052 2731 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:08:30.159204 2731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "e2305e11e279480bb5be780cb9570271"
format_stamp: "Formatted at 2026-08-12 03:08:28 on dist-test-slave-zpfg"
server_key: "67fa6ee90db408889eaff64efb619ca5"
server_key_iv: "5e2fd0f3025d773c7636d82a4934f7fd"
server_key_version: "encryptionkey@0"
I20260812 03:08:30.159821 2731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:30.178612 2731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:30.183190 2731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:30.183445 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:30.184123 2731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:30.185051 2731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:30.185124 2731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:30.185189 2731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:30.185241 2731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:30.196862 2731 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:36363
I20260812 03:08:30.196926 2866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:36363 every 8 connection(s)
I20260812 03:08:30.198153 2731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:08:30.198596 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 2731
I20260812 03:08:30.198791 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 2024
I20260812 03:08:30.210304 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:39537
--local_ip_for_outbound_sockets=127.26.250.131
--tserver_master_addrs=127.26.250.190:39831
--webserver_port=37897
--webserver_interface=127.26.250.131
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.131.keytab
--principal=oryx/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:39227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.26.250.190:39831,127.26.250.189:42549
--tserver_master_addrs=127.26.250.190:39831,127.26.250.189:42549,127.26.250.188:39479 with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:08:30 dist-test-slave-zpfg krb5kdc[1611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504110, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.188@KRBTEST.COM
Aug 12 03:08:30 dist-test-slave-zpfg krb5kdc[1611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504110, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM
I20260812 03:08:30.216974 2730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.200615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:44557 (local address 127.26.250.188:39479)
0812 03:08:30.201213 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:30.201217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:30.202021 (+ 804us) server_negotiation.cc:407] Connection header received
0812 03:08:30.203759 (+ 1738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:30.203762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:30.203824 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:30.203941 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:30.205240 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.205790 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.207894 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.208300 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.208397 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.212046 (+ 3649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:30.212052 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:30.212058 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:30.212098 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:30.213955 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:30.214710 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:30.214715 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:30.214718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:30.214826 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:30.215284 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:30.215290 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:30.215294 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:30.215605 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:30.215711 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:30.216526 (+ 815us) server_negotiation.cc:299] Negotiation successful
0812 03:08:30.216707 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479}
Aug 12 03:08:30 dist-test-slave-zpfg krb5kdc[1611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504110, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.189@KRBTEST.COM
I20260812 03:08:30.218148 1702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.206828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:40437 (local address 127.26.250.190:39831)
0812 03:08:30.206963 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:30.206968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:30.207536 (+ 568us) server_negotiation.cc:407] Connection header received
0812 03:08:30.207727 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:30.207733 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:30.207793 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:30.207849 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:30.208719 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.209596 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.210785 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.211155 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.211276 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.214108 (+ 2832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:30.214112 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:30.214115 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:30.214146 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:30.215932 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:30.216628 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:30.216632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:30.216633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:30.216726 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:30.217079 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:30.217083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:30.217084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:30.217360 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:30.217437 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:30.217853 (+ 416us) server_negotiation.cc:299] Negotiation successful
0812 03:08:30.217997 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:08:30.219169 2873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.206562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.26.250.130:40437)
0812 03:08:30.207437 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:30.207454 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:30.207614 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:30.208018 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:30.208023 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:30.208285 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:30.208558 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:30.208579 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.209752 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.209755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:30.210307 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:30.210314 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.211302 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.211308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:30.211443 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.212457 (+ 1014us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:30.212472 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:30.213931 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:30.216070 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:30.216076 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:30.216079 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:30.216506 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:30.216860 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:30.216865 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:30.216866 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:30.216960 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:30.217533 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:30.217537 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:30.217647 (+ 110us) client_negotiation.cc:769] Sending connection context
0812 03:08:30.218325 (+ 678us) client_negotiation.cc:240] Negotiation successful
0812 03:08:30.218507 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"mutex_wait_us":78,"spinlock_wait_cycles":2304,"thread_start_us":541,"threads_started":1}
I20260812 03:08:30.219184 2871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.201144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:39479 (local address 127.26.250.130:44557)
0812 03:08:30.201834 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:30.201902 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:30.203508 (+ 1606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:30.204143 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:30.204151 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:30.204547 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:30.205074 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:30.205088 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.205931 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.205935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:30.207747 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:30.207760 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.208616 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.208619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:30.208837 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.209725 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:30.209751 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:30.211850 (+ 2099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:30.214103 (+ 2253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:30.214114 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:30.214136 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:30.214555 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:30.214938 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:30.214944 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:30.214949 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:30.215159 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:30.215788 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:30.215798 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:30.216212 (+ 414us) client_negotiation.cc:769] Sending connection context
0812 03:08:30.217252 (+ 1040us) client_negotiation.cc:240] Negotiation successful
0812 03:08:30.217686 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"spinlock_wait_cycles":9984,"thread_start_us":223,"threads_started":1}
I20260812 03:08:30.220878 2868 heartbeater.cc:347] Connected to a master server at 127.26.250.190:39831
I20260812 03:08:30.220880 2867 heartbeater.cc:347] Connected to a master server at 127.26.250.188:39479
I20260812 03:08:30.221174 2867 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:30.221174 2868 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:30.221963 2868 heartbeater.cc:511] Master 127.26.250.190:39831 requested a full tablet report, sending...
I20260812 03:08:30.221959 2867 heartbeater.cc:511] Master 127.26.250.188:39479 requested a full tablet report, sending...
I20260812 03:08:30.223379 1658 ts_manager.cc:194] Re-registered known tserver with Master: e2305e11e279480bb5be780cb9570271 (127.26.250.130:36363)
I20260812 03:08:30.223749 2529 ts_manager.cc:194] Registered new tserver with Master: e2305e11e279480bb5be780cb9570271 (127.26.250.130:36363)
I20260812 03:08:30.224151 1658 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.250.130@KRBTEST.COM'} at 127.26.250.130:40437
I20260812 03:08:30.226286 2872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.206467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:42549 (local address 127.26.250.130:59639)
0812 03:08:30.207282 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:30.207297 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:30.207442 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:30.207802 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:30.207807 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:30.208140 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:30.208368 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:30.208376 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.212275 (+ 3899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.212277 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:30.213273 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:30.213288 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.214153 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.214157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:30.214259 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.216147 (+ 1888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:30.216166 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:30.218811 (+ 2645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:30.221847 (+ 3036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:30.221850 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:30.221853 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:30.222286 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:30.222735 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:30.222739 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:30.222743 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:30.222860 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:30.225513 (+ 2653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:30.225520 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:30.225663 (+ 143us) client_negotiation.cc:769] Sending connection context
0812 03:08:30.225787 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 03:08:30.226003 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"mutex_wait_us":3,"spinlock_wait_cycles":24320,"thread_start_us":559,"threads_started":1}
I20260812 03:08:30.226958 2457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.206849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:59639 (local address 127.26.250.189:42549)
0812 03:08:30.206981 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:30.206987 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:30.207376 (+ 389us) server_negotiation.cc:407] Connection header received
0812 03:08:30.207548 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:30.207551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:30.207603 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:30.207656 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:30.208519 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.209651 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.213428 (+ 3777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.213810 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.213929 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.218991 (+ 5062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:30.218995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:30.218998 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:30.219029 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:30.221579 (+ 2550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:30.222501 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:30.222505 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:30.222507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:30.222624 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:30.224921 (+ 2297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:30.224925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:30.224927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:30.225376 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:30.225511 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:30.226575 (+ 1064us) server_negotiation.cc:299] Negotiation successful
0812 03:08:30.226746 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 03:08:30.227965 2869 heartbeater.cc:347] Connected to a master server at 127.26.250.189:42549
I20260812 03:08:30.228070 2869 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:30.228291 2869 heartbeater.cc:511] Master 127.26.250.189:42549 requested a full tablet report, sending...
I20260812 03:08:30.229085 2321 ts_manager.cc:194] Registered new tserver with Master: e2305e11e279480bb5be780cb9570271 (127.26.250.130:36363)
I20260812 03:08:30.344558 2875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.339945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:39479 (local address 127.0.0.1:40398)
0812 03:08:30.340363 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:30.340380 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:30.340516 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:30.340862 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:30.340865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:30.340886 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:30.340991 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:30.340996 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.342109 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.342112 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:30.343069 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:30.343081 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.344080 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.344085 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:30.344197 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.344222 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 03:08:30.344289 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 03:08:30.344366 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":90,"threads_started":1}
I20260812 03:08:30.344573 2730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.340122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40398 (local address 127.26.250.188:39479)
0812 03:08:30.340300 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:30.340306 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:30.340482 (+ 176us) server_negotiation.cc:407] Connection header received
0812 03:08:30.340636 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:30.340640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:30.340717 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:30.340831 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:30.341125 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.341958 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.343211 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.343921 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.344065 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.344209 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:30.344331 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 03:08:30.344428 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 03:08:30.345381 2540 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 1 LEARNER]: Deduplicated request from leader. Original: 1.5->[1.6-1.7] Dedup: 1.7->[]
I20260812 03:08:30.345798 2570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [sys.catalog]: SysCatalogTable state changed. Reason: New leader c767d9b346ba4b9d9b221992229a332c. Latest consensus state: current_term: 1 leader_uuid: "c767d9b346ba4b9d9b221992229a332c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: NON_VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: true } } }
I20260812 03:08:30.346012 2570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [sys.catalog]: This master's current role is: LEARNER
I20260812 03:08:30.346511 2486 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c: attempting to promote NON_VOTER 63499cf6e1c048d6be211a8a4fdaafba to VOTER
I20260812 03:08:30.346940 2486 consensus_queue.cc:237] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: false } }
I20260812 03:08:30.348453 2332 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [term 1 FOLLOWER]: Refusing update from remote peer c767d9b346ba4b9d9b221992229a332c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260812 03:08:30.348578 2540 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 1 LEARNER]: Refusing update from remote peer c767d9b346ba4b9d9b221992229a332c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260812 03:08:30.348886 2880 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [LEADER]: Connected to new peer: Peer: permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 03:08:30.349359 2486 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [LEADER]: Connected to new peer: Peer: permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 03:08:30.350979 2880 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 63499cf6e1c048d6be211a8a4fdaafba (127.26.250.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: false } } }
I20260812 03:08:30.351455 2332 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 63499cf6e1c048d6be211a8a4fdaafba (127.26.250.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: false } } }
W20260812 03:08:30.351951 2874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
I20260812 03:08:30.352100 2540 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 63499cf6e1c048d6be211a8a4fdaafba (127.26.250.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: false } } }
W20260812 03:08:30.352341 2874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:30.352423 2874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 03:08:30.352674 2880 sys_catalog.cc:455] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c767d9b346ba4b9d9b221992229a332c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: false } } }
I20260812 03:08:30.352798 2880 sys_catalog.cc:458] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:30.353008 2880 sys_catalog.cc:455] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c767d9b346ba4b9d9b221992229a332c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: false } } }
I20260812 03:08:30.353094 2880 sys_catalog.cc:458] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:30.353199 2880 sys_catalog.cc:455] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c767d9b346ba4b9d9b221992229a332c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: false } } }
I20260812 03:08:30.353322 2880 sys_catalog.cc:458] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:30.353609 2570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c767d9b346ba4b9d9b221992229a332c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: false } } }
I20260812 03:08:30.353739 2570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:08:30.354254 2883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c767d9b346ba4b9d9b221992229a332c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: false } } }
I20260812 03:08:30.354395 2883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 03:08:30.357431 2874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:30.357499 2874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:30.357613 2874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:08:30.363711 2874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:39227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.131.keytab
--principal=oryx/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:39537
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=37897
--tserver_master_addrs=127.26.250.190:39831,127.26.250.189:42549,127.26.250.188:39479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:30.365046 2874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:30.366200 2874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:30.376708 2886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:30.376742 2889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:30.376760 2887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:30.377079 2874 server_base.cc:1034] running on GCE node
I20260812 03:08:30.377568 2874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:30.378271 2874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:30.379539 2874 hybrid_clock.cc:648] HybridClock initialized: now 1786504110379518 us; error 27 us; skew 500 ppm
Aug 12 03:08:30 dist-test-slave-zpfg krb5kdc[1611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504110, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:30.382880 2874 init.cc:377] Logged in from keytab as oryx/127.26.250.131@KRBTEST.COM (short username oryx)
I20260812 03:08:30.383780 2874 webserver.cc:460] Webserver started at http://127.26.250.131:37897/ using document root <none> and password file <none>
I20260812 03:08:30.384336 2874 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:30.384413 2874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:30.388491 2874 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:08:30.390913 2898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:30.392407 2874 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:08:30.392513 2874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "ac62277b98f34ff58e5d13dee22266bc"
format_stamp: "Formatted at 2026-08-12 03:08:28 on dist-test-slave-zpfg"
server_key: "963c6e3512606289931356108985deb9"
server_key_iv: "35b1b6e793b333815644355ebd4af040"
server_key_version: "encryptionkey@0"
I20260812 03:08:30.393060 2874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:30.421595 2874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:30.424764 2874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:30.424945 2874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:30.425581 2874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:30.426528 2874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:30.426580 2874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:30.426622 2874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:30.426645 2874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:30.436401 2874 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:39537
I20260812 03:08:30.436489 3019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:39537 every 8 connection(s)
I20260812 03:08:30.437621 2874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:08:30.441644 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 2874
Aug 12 03:08:30 dist-test-slave-zpfg krb5kdc[1611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504110, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM
Aug 12 03:08:30 dist-test-slave-zpfg krb5kdc[1611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504110, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.189@KRBTEST.COM
I20260812 03:08:30.457211 3034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.444054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59776 (local address 127.26.250.189:42549)
0812 03:08:30.444500 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:30.444505 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:30.444973 (+ 468us) server_negotiation.cc:407] Connection header received
0812 03:08:30.445019 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:30.445022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:30.445072 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:30.445151 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:30.446361 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.447179 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.448371 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.448707 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.448809 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.450688 (+ 1879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:30.450693 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:30.450700 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:30.450735 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:30.452633 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:30.454310 (+ 1677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:30.454314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:30.454316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:30.454445 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:30.454762 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:30.454765 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:30.454766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:30.455002 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:30.455134 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:30.456853 (+ 1719us) server_negotiation.cc:299] Negotiation successful
0812 03:08:30.457015 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":62,"threads_started":1}
I20260812 03:08:30.458426 2457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.439811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:53075 (local address 127.26.250.189:42549)
0812 03:08:30.439967 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:30.439973 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:30.441501 (+ 1528us) server_negotiation.cc:407] Connection header received
0812 03:08:30.442630 (+ 1129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:30.442634 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:30.442703 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:30.442767 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:30.445014 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.445784 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.447733 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.448093 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.448178 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.453407 (+ 5229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:30.453410 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:30.453413 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:30.453438 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:30.455213 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:30.456347 (+ 1134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:30.456351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:30.456353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:30.456442 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:30.456928 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:30.456931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:30.456933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:30.457128 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:30.457209 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:30.458089 (+ 880us) server_negotiation.cc:299] Negotiation successful
0812 03:08:30.458249 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 03:08:30.458449 1702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.439854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:42683 (local address 127.26.250.190:39831)
0812 03:08:30.439975 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:30.439981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:30.441098 (+ 1117us) server_negotiation.cc:407] Connection header received
0812 03:08:30.442611 (+ 1513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:30.442615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:30.442672 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:30.442728 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:30.445021 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.445889 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.447719 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.448082 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.448154 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.452535 (+ 4381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:30.452539 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:30.452542 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:30.452575 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:30.455304 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:30.456359 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:30.456365 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:30.456370 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:30.456464 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:30.456903 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:30.456907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:30.456910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:30.457144 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:30.457236 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:30.458100 (+ 864us) server_negotiation.cc:299] Negotiation successful
0812 03:08:30.458227 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Aug 12 03:08:30 dist-test-slave-zpfg krb5kdc[1611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504110, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.188@KRBTEST.COM
I20260812 03:08:30.460737 3037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.445757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40414 (local address 127.26.250.188:39479)
0812 03:08:30.446322 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:30.446327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:30.448227 (+ 1900us) server_negotiation.cc:407] Connection header received
0812 03:08:30.448279 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:30.448284 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:30.448354 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:30.448436 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:30.451370 (+ 2934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.452116 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.453319 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.453753 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.453866 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.456420 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:30.456427 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:30.456431 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:30.456456 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:30.458514 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:30.459052 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:30.459057 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:30.459059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:30.459148 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:30.459467 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:30.459473 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:30.459474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:30.459683 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:30.459802 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:30.460404 (+ 602us) server_negotiation.cc:299] Negotiation successful
0812 03:08:30.460529 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":105,"threads_started":1}
I20260812 03:08:30.460958 3026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.440311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:42549 (local address 127.26.250.131:53075)
0812 03:08:30.441392 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:30.441410 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:30.442313 (+ 903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:30.443158 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:30.443169 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:30.444174 (+ 1005us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:30.444842 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:30.444855 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.445986 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.445989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:30.447589 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:30.447602 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.448387 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.448393 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:30.448739 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.449700 (+ 961us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:30.449715 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:30.453241 (+ 3526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:30.455506 (+ 2265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:30.455512 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:30.455537 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:30.456233 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:30.456549 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:30.456551 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:30.456553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:30.456770 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:30.457286 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:30.457294 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:30.457791 (+ 497us) client_negotiation.cc:769] Sending connection context
0812 03:08:30.458639 (+ 848us) client_negotiation.cc:240] Negotiation successful
0812 03:08:30.459049 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":954,"mutex_wait_us":330,"spinlock_wait_cycles":67968,"thread_start_us":177,"threads_started":1}
I20260812 03:08:30.461973 3025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.440309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.26.250.131:42683)
0812 03:08:30.440900 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:30.440945 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:30.442327 (+ 1382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:30.443158 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:30.443169 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:30.443899 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:30.444831 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:30.444853 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.446114 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.446119 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:30.447581 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:30.447591 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.448399 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.448404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:30.448646 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.449334 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:30.449353 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:30.452312 (+ 2959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:30.455436 (+ 3124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:30.455444 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:30.455716 (+ 272us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:30.456229 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:30.456581 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:30.456587 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:30.456589 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:30.456759 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:30.457603 (+ 844us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:30.457607 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:30.457795 (+ 188us) client_negotiation.cc:769] Sending connection context
0812 03:08:30.459067 (+ 1272us) client_negotiation.cc:240] Negotiation successful
0812 03:08:30.459236 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":171,"spinlock_wait_cycles":19456,"thread_start_us":770,"threads_started":1}
I20260812 03:08:30.462618 3023 heartbeater.cc:347] Connected to a master server at 127.26.250.189:42549
I20260812 03:08:30.462940 3023 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:30.463517 3023 heartbeater.cc:511] Master 127.26.250.189:42549 requested a full tablet report, sending...
I20260812 03:08:30.463909 3035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.445093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57440 (local address 127.26.250.190:39831)
0812 03:08:30.446866 (+ 1773us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:30.446876 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:30.446889 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:08:30.446922 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:30.446925 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:30.446969 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:30.447046 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:30.448562 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.449409 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.453350 (+ 3941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.453723 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.453830 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.458620 (+ 4790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:30.458624 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:30.458627 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:30.458652 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:30.460699 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:30.461245 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:30.461249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:30.461250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:30.461362 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:30.461684 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:30.461699 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:30.461704 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:30.461954 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:30.462124 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:30.463569 (+ 1445us) server_negotiation.cc:299] Negotiation successful
0812 03:08:30.463724 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1688,"thread_start_us":93,"threads_started":1}
I20260812 03:08:30.462618 3022 heartbeater.cc:347] Connected to a master server at 127.26.250.190:39831
I20260812 03:08:30.464246 3022 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:30.464455 3022 heartbeater.cc:511] Master 127.26.250.190:39831 requested a full tablet report, sending...
I20260812 03:08:30.465305 1658 ts_manager.cc:194] Re-registered known tserver with Master: ac62277b98f34ff58e5d13dee22266bc (127.26.250.131:39537)
I20260812 03:08:30.465655 2321 ts_manager.cc:194] Registered new tserver with Master: ac62277b98f34ff58e5d13dee22266bc (127.26.250.131:39537)
I20260812 03:08:30.466039 1658 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.250.131@KRBTEST.COM'} at 127.26.250.131:42683
I20260812 03:08:30.466172 3027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.441584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:39479 (local address 127.26.250.131:39867)
0812 03:08:30.441857 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:30.441872 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:30.442333 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:30.443158 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:30.443169 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:30.443882 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:30.448632 (+ 4750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:30.448641 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.452253 (+ 3612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.452256 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:30.454428 (+ 2172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:30.454439 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.455172 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.455178 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:30.455280 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.455924 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:30.455938 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:30.461864 (+ 5926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:30.463919 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:30.463922 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:30.463924 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:30.464563 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:30.464909 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:30.464914 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:30.464919 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:30.465022 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:30.465586 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:30.465588 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:30.465685 (+ 97us) client_negotiation.cc:769] Sending connection context
0812 03:08:30.465826 (+ 141us) client_negotiation.cc:240] Negotiation successful
0812 03:08:30.465997 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":245,"spinlock_wait_cycles":3712,"thread_start_us":99,"threads_started":1}
I20260812 03:08:30.466189 2730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.442052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:39867 (local address 127.26.250.188:39479)
0812 03:08:30.442185 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:30.442190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:30.442207 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:08:30.442612 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:30.442615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:30.442672 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:30.442728 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:30.450463 (+ 7735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.451631 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.454566 (+ 2935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.454909 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.454985 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.462044 (+ 7059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:30.462050 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:30.462053 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:30.462077 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:30.463806 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:30.464699 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:30.464703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:30.464704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:30.464793 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:30.465140 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:30.465143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:30.465146 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:30.465400 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:30.465489 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:30.465911 (+ 422us) server_negotiation.cc:299] Negotiation successful
0812 03:08:30.466033 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:08:30.467157 3020 heartbeater.cc:347] Connected to a master server at 127.26.250.188:39479
I20260812 03:08:30.467284 3020 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:30.467747 3020 heartbeater.cc:511] Master 127.26.250.188:39479 requested a full tablet report, sending...
I20260812 03:08:30.468511 2529 ts_manager.cc:194] Registered new tserver with Master: ac62277b98f34ff58e5d13dee22266bc (127.26.250.131:39537)
I20260812 03:08:30.468377 1658 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57440:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:08:30.471714 1658 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:08:30.491607 2875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.485550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:36363 (local address 127.0.0.1:48316)
0812 03:08:30.485668 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:30.485684 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:30.485766 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:30.487103 (+ 1337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:30.487106 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:30.487177 (+ 71us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:30.487335 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:30.487342 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.489277 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.489280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:30.490078 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:30.490087 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.491227 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.491231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:30.491315 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.491336 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 03:08:30.491389 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 03:08:30.491454 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":27}
I20260812 03:08:30.491715 3045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.485736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48316 (local address 127.26.250.130:36363)
0812 03:08:30.486437 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:30.486455 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:30.486484 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:08:30.486581 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:30.486587 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:30.486870 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:30.487514 (+ 644us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:30.487779 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.489092 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.490217 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.491115 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.491249 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.491398 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:30.491457 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 03:08:30.491553 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":180,"threads_started":1}
I20260812 03:08:30.492443 3049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.486674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53988 (local address 127.26.250.129:46261)
0812 03:08:30.487081 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:30.487100 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:30.487124 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:08:30.487206 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:30.487211 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:30.487432 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:30.487602 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:30.488039 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.488788 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.490315 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.491108 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.491249 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.491398 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:30.492187 (+ 789us) server_negotiation.cc:299] Negotiation successful
0812 03:08:30.492292 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":77,"threads_started":1}
I20260812 03:08:30.492865 3046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.485554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:39537 (local address 127.0.0.1:54334)
0812 03:08:30.486143 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:30.486160 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:30.486261 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:30.487343 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:30.487347 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:30.487366 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:30.487496 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:30.487503 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.489148 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.489151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:30.490380 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:30.490389 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.492390 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.492393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:30.492585 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.492617 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 03:08:30.492691 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 03:08:30.492759 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"mutex_wait_us":329,"thread_start_us":96,"threads_started":1}
I20260812 03:08:30.493019 3047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.486164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:46261 (local address 127.0.0.1:53988)
0812 03:08:30.486686 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:30.486701 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:30.486798 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:30.487769 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:30.487772 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:30.487789 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:30.487898 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:30.487905 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.488928 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.488933 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:30.490204 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:30.490215 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.491969 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.491972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:30.492038 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.492057 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 03:08:30.492147 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 03:08:30.492221 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"mutex_wait_us":23,"thread_start_us":106,"threads_started":1}
I20260812 03:08:30.494752 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.486232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54334 (local address 127.26.250.131:39537)
0812 03:08:30.486829 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:30.486843 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:30.486869 (+ 26us) server_negotiation.cc:407] Connection header received
0812 03:08:30.486952 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:30.486962 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:30.487176 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:30.487743 (+ 567us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:30.487914 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.489021 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.491467 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.492284 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.492413 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.492539 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:30.494470 (+ 1931us) server_negotiation.cc:299] Negotiation successful
0812 03:08:30.494598 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":124,"threads_started":1}
I20260812 03:08:30.495071 2781 tablet_service.cc:1467] Processing CreateTablet for tablet 2164dc8003cb4fec940dd8bfc03184a1 (DEFAULT_TABLE table=test-table [id=454c64e72cf64162812c8b8e9ebaa42b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:30.496469 2781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2164dc8003cb4fec940dd8bfc03184a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:30.496024 2653 tablet_service.cc:1467] Processing CreateTablet for tablet 2164dc8003cb4fec940dd8bfc03184a1 (DEFAULT_TABLE table=test-table [id=454c64e72cf64162812c8b8e9ebaa42b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:30.497318 2653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2164dc8003cb4fec940dd8bfc03184a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:30.497413 2949 tablet_service.cc:1467] Processing CreateTablet for tablet 2164dc8003cb4fec940dd8bfc03184a1 (DEFAULT_TABLE table=test-table [id=454c64e72cf64162812c8b8e9ebaa42b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:30.498332 2949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2164dc8003cb4fec940dd8bfc03184a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:30.506258 3051 tablet_bootstrap.cc:492] T 2164dc8003cb4fec940dd8bfc03184a1 P f29103f78e19408087681f54d256c8f7: Bootstrap starting.
I20260812 03:08:30.506258 3052 tablet_bootstrap.cc:492] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271: Bootstrap starting.
I20260812 03:08:30.506836 3050 tablet_bootstrap.cc:492] T 2164dc8003cb4fec940dd8bfc03184a1 P ac62277b98f34ff58e5d13dee22266bc: Bootstrap starting.
I20260812 03:08:30.508661 3050 tablet_bootstrap.cc:654] T 2164dc8003cb4fec940dd8bfc03184a1 P ac62277b98f34ff58e5d13dee22266bc: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:30.508694 3051 tablet_bootstrap.cc:654] T 2164dc8003cb4fec940dd8bfc03184a1 P f29103f78e19408087681f54d256c8f7: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:30.508749 3052 tablet_bootstrap.cc:654] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:30.509452 3050 log.cc:824] T 2164dc8003cb4fec940dd8bfc03184a1 P ac62277b98f34ff58e5d13dee22266bc: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:30.509536 3052 log.cc:824] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:30.509536 3051 log.cc:824] T 2164dc8003cb4fec940dd8bfc03184a1 P f29103f78e19408087681f54d256c8f7: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:30.511809 3052 tablet_bootstrap.cc:492] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271: No bootstrap required, opened a new log
I20260812 03:08:30.511886 3051 tablet_bootstrap.cc:492] T 2164dc8003cb4fec940dd8bfc03184a1 P f29103f78e19408087681f54d256c8f7: No bootstrap required, opened a new log
I20260812 03:08:30.511920 3050 tablet_bootstrap.cc:492] T 2164dc8003cb4fec940dd8bfc03184a1 P ac62277b98f34ff58e5d13dee22266bc: No bootstrap required, opened a new log
I20260812 03:08:30.512060 3052 ts_tablet_manager.cc:1397] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:08:30.512081 3050 ts_tablet_manager.cc:1397] T 2164dc8003cb4fec940dd8bfc03184a1 P ac62277b98f34ff58e5d13dee22266bc: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 03:08:30.512082 3051 ts_tablet_manager.cc:1397] T 2164dc8003cb4fec940dd8bfc03184a1 P f29103f78e19408087681f54d256c8f7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:08:30.516419 3051 raft_consensus.cc:348] T 2164dc8003cb4fec940dd8bfc03184a1 P f29103f78e19408087681f54d256c8f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac62277b98f34ff58e5d13dee22266bc" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39537 } } peers { permanent_uuid: "f29103f78e19408087681f54d256c8f7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46261 } } peers { permanent_uuid: "e2305e11e279480bb5be780cb9570271" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 36363 } }
I20260812 03:08:30.516734 3051 raft_consensus.cc:374] T 2164dc8003cb4fec940dd8bfc03184a1 P f29103f78e19408087681f54d256c8f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:30.516793 3051 raft_consensus.cc:729] T 2164dc8003cb4fec940dd8bfc03184a1 P f29103f78e19408087681f54d256c8f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f29103f78e19408087681f54d256c8f7, State: Initialized, Role: FOLLOWER
I20260812 03:08:30.516920 3050 raft_consensus.cc:348] T 2164dc8003cb4fec940dd8bfc03184a1 P ac62277b98f34ff58e5d13dee22266bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac62277b98f34ff58e5d13dee22266bc" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39537 } } peers { permanent_uuid: "f29103f78e19408087681f54d256c8f7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46261 } } peers { permanent_uuid: "e2305e11e279480bb5be780cb9570271" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 36363 } }
I20260812 03:08:30.517261 3050 raft_consensus.cc:374] T 2164dc8003cb4fec940dd8bfc03184a1 P ac62277b98f34ff58e5d13dee22266bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:30.517318 3050 raft_consensus.cc:729] T 2164dc8003cb4fec940dd8bfc03184a1 P ac62277b98f34ff58e5d13dee22266bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac62277b98f34ff58e5d13dee22266bc, State: Initialized, Role: FOLLOWER
I20260812 03:08:30.517421 3051 consensus_queue.cc:260] T 2164dc8003cb4fec940dd8bfc03184a1 P f29103f78e19408087681f54d256c8f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac62277b98f34ff58e5d13dee22266bc" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39537 } } peers { permanent_uuid: "f29103f78e19408087681f54d256c8f7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46261 } } peers { permanent_uuid: "e2305e11e279480bb5be780cb9570271" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 36363 } }
I20260812 03:08:30.517781 3050 consensus_queue.cc:260] T 2164dc8003cb4fec940dd8bfc03184a1 P ac62277b98f34ff58e5d13dee22266bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac62277b98f34ff58e5d13dee22266bc" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39537 } } peers { permanent_uuid: "f29103f78e19408087681f54d256c8f7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46261 } } peers { permanent_uuid: "e2305e11e279480bb5be780cb9570271" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 36363 } }
I20260812 03:08:30.517755 3052 raft_consensus.cc:348] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac62277b98f34ff58e5d13dee22266bc" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39537 } } peers { permanent_uuid: "f29103f78e19408087681f54d256c8f7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46261 } } peers { permanent_uuid: "e2305e11e279480bb5be780cb9570271" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 36363 } }
I20260812 03:08:30.518106 3052 raft_consensus.cc:374] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:30.518193 3052 raft_consensus.cc:729] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2305e11e279480bb5be780cb9570271, State: Initialized, Role: FOLLOWER
I20260812 03:08:30.518590 3051 ts_tablet_manager.cc:1428] T 2164dc8003cb4fec940dd8bfc03184a1 P f29103f78e19408087681f54d256c8f7: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260812 03:08:30.518851 3050 ts_tablet_manager.cc:1428] T 2164dc8003cb4fec940dd8bfc03184a1 P ac62277b98f34ff58e5d13dee22266bc: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 03:08:30.518767 3052 consensus_queue.cc:260] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac62277b98f34ff58e5d13dee22266bc" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39537 } } peers { permanent_uuid: "f29103f78e19408087681f54d256c8f7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46261 } } peers { permanent_uuid: "e2305e11e279480bb5be780cb9570271" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 36363 } }
I20260812 03:08:30.519783 3052 ts_tablet_manager.cc:1428] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260812 03:08:30.520342 3022 heartbeater.cc:503] Master 127.26.250.190:39831 was elected leader, sending a full tablet report...
I20260812 03:08:30.520377 2868 heartbeater.cc:503] Master 127.26.250.190:39831 was elected leader, sending a full tablet report...
I20260812 03:08:30.524591 2723 heartbeater.cc:503] Master 127.26.250.190:39831 was elected leader, sending a full tablet report...
I20260812 03:08:30.543519 3060 raft_consensus.cc:482] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:08:30.543726 3060 raft_consensus.cc:504] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac62277b98f34ff58e5d13dee22266bc" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39537 } } peers { permanent_uuid: "f29103f78e19408087681f54d256c8f7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46261 } } peers { permanent_uuid: "e2305e11e279480bb5be780cb9570271" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 36363 } }
I20260812 03:08:30.545006 3060 leader_election.cc:288] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ac62277b98f34ff58e5d13dee22266bc (127.26.250.131:39537), f29103f78e19408087681f54d256c8f7 (127.26.250.129:46261)
I20260812 03:08:30.549026 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.545418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:50357 (local address 127.26.250.131:39537)
0812 03:08:30.545567 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:30.545572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:30.545586 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:08:30.545642 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:30.545645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:30.545698 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:30.545786 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:30.546094 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.546920 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.547979 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.548400 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.548539 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.548597 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:30.548787 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 03:08:30.548872 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:08:30.549023 2872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.545182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:39537 (local address 127.26.250.130:50357)
0812 03:08:30.545338 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:30.545356 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:30.545449 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:30.545792 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:30.545795 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:30.545808 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:30.545948 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:30.545957 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.547031 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.547037 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:30.547822 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:30.547857 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.548534 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.548538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:30.548654 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.548673 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 03:08:30.548755 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 03:08:30.548858 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 03:08:30.549348 2873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.545506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:46261 (local address 127.26.250.130:42877)
0812 03:08:30.545605 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:30.545617 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:30.545670 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:30.546537 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:30.546541 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:30.546562 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:30.546686 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:30.546692 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.547486 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.547490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:30.548250 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:30.548260 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.548915 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.548919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:30.549028 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.549053 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 03:08:30.549127 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 03:08:30.549223 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":1}
I20260812 03:08:30.549611 3049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.545993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:42877 (local address 127.26.250.129:46261)
0812 03:08:30.546172 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:30.546178 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:30.546194 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:08:30.546268 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:30.546273 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:30.546416 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:30.546502 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:30.546805 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.547370 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.548387 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.548791 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.548926 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.548979 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:30.549370 (+ 391us) server_negotiation.cc:299] Negotiation successful
0812 03:08:30.549460 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 03:08:30.549821 2970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2164dc8003cb4fec940dd8bfc03184a1" candidate_uuid: "e2305e11e279480bb5be780cb9570271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac62277b98f34ff58e5d13dee22266bc" is_pre_election: true
I20260812 03:08:30.550249 2970 raft_consensus.cc:2454] T 2164dc8003cb4fec940dd8bfc03184a1 P ac62277b98f34ff58e5d13dee22266bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2305e11e279480bb5be780cb9570271 in term 0.
I20260812 03:08:30.550504 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2164dc8003cb4fec940dd8bfc03184a1" candidate_uuid: "e2305e11e279480bb5be780cb9570271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f29103f78e19408087681f54d256c8f7" is_pre_election: true
I20260812 03:08:30.550786 2673 raft_consensus.cc:2454] T 2164dc8003cb4fec940dd8bfc03184a1 P f29103f78e19408087681f54d256c8f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2305e11e279480bb5be780cb9570271 in term 0.
I20260812 03:08:30.550889 2749 leader_election.cc:302] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac62277b98f34ff58e5d13dee22266bc, e2305e11e279480bb5be780cb9570271; no voters:
I20260812 03:08:30.551262 3060 raft_consensus.cc:2781] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:08:30.551400 3060 raft_consensus.cc:482] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:08:30.551486 3060 raft_consensus.cc:3037] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:30.552623 3060 raft_consensus.cc:504] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac62277b98f34ff58e5d13dee22266bc" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39537 } } peers { permanent_uuid: "f29103f78e19408087681f54d256c8f7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46261 } } peers { permanent_uuid: "e2305e11e279480bb5be780cb9570271" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 36363 } }
I20260812 03:08:30.552986 3060 leader_election.cc:288] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271 [CANDIDATE]: Term 1 election: Requested vote from peers ac62277b98f34ff58e5d13dee22266bc (127.26.250.131:39537), f29103f78e19408087681f54d256c8f7 (127.26.250.129:46261)
I20260812 03:08:30.553347 2970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2164dc8003cb4fec940dd8bfc03184a1" candidate_uuid: "e2305e11e279480bb5be780cb9570271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac62277b98f34ff58e5d13dee22266bc"
I20260812 03:08:30.553488 2970 raft_consensus.cc:3037] T 2164dc8003cb4fec940dd8bfc03184a1 P ac62277b98f34ff58e5d13dee22266bc [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:30.553535 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2164dc8003cb4fec940dd8bfc03184a1" candidate_uuid: "e2305e11e279480bb5be780cb9570271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f29103f78e19408087681f54d256c8f7"
I20260812 03:08:30.553700 2673 raft_consensus.cc:3037] T 2164dc8003cb4fec940dd8bfc03184a1 P f29103f78e19408087681f54d256c8f7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:30.554710 2970 raft_consensus.cc:2454] T 2164dc8003cb4fec940dd8bfc03184a1 P ac62277b98f34ff58e5d13dee22266bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2305e11e279480bb5be780cb9570271 in term 1.
I20260812 03:08:30.555003 2673 raft_consensus.cc:2454] T 2164dc8003cb4fec940dd8bfc03184a1 P f29103f78e19408087681f54d256c8f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2305e11e279480bb5be780cb9570271 in term 1.
I20260812 03:08:30.555063 2749 leader_election.cc:302] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac62277b98f34ff58e5d13dee22266bc, e2305e11e279480bb5be780cb9570271; no voters:
I20260812 03:08:30.555238 3060 raft_consensus.cc:2781] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:30.555523 3060 raft_consensus.cc:686] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271 [term 1 LEADER]: Becoming Leader. State: Replica: e2305e11e279480bb5be780cb9570271, State: Running, Role: LEADER
I20260812 03:08:30.555898 3060 consensus_queue.cc:237] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac62277b98f34ff58e5d13dee22266bc" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39537 } } peers { permanent_uuid: "f29103f78e19408087681f54d256c8f7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46261 } } peers { permanent_uuid: "e2305e11e279480bb5be780cb9570271" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 36363 } }
I20260812 03:08:30.559662 1658 catalog_manager.cc:5495] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2305e11e279480bb5be780cb9570271 (127.26.250.130). New cstate: current_term: 1 leader_uuid: "e2305e11e279480bb5be780cb9570271" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac62277b98f34ff58e5d13dee22266bc" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f29103f78e19408087681f54d256c8f7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2305e11e279480bb5be780cb9570271" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 36363 } health_report { overall_health: HEALTHY } } }
I20260812 03:08:30.592550 3045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.588118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48318 (local address 127.26.250.130:36363)
0812 03:08:30.588276 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:30.588283 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:30.588299 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:08:30.588395 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:30.588397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:30.588452 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:30.588572 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:08:30.588882 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.589386 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.590712 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.591093 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.591194 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.591551 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:08:30.592142 (+ 591us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:08:30.592327 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:30.592359 (+ 32us) server_negotiation.cc:299] Negotiation successful
0812 03:08:30.592422 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 03:08:30.599663 2673 raft_consensus.cc:1261] T 2164dc8003cb4fec940dd8bfc03184a1 P f29103f78e19408087681f54d256c8f7 [term 1 FOLLOWER]: Refusing update from remote peer e2305e11e279480bb5be780cb9570271: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:30.599661 2970 raft_consensus.cc:1261] T 2164dc8003cb4fec940dd8bfc03184a1 P ac62277b98f34ff58e5d13dee22266bc [term 1 FOLLOWER]: Refusing update from remote peer e2305e11e279480bb5be780cb9570271: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:30.600526 3060 consensus_queue.cc:1035] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f29103f78e19408087681f54d256c8f7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:30.600770 3061 consensus_queue.cc:1035] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac62277b98f34ff58e5d13dee22266bc" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:30.611415 3065 mvcc.cc:205] Tried to move back new op lower bound from 7317520837007265792 to 7317520836840161280. Current Snapshot: MvccSnapshot[applied={T|T < 7317520837007265792}]
I20260812 03:08:30.624018 1661 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:30.624202 1661 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:30.629220 1661 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c767d9b346ba4b9d9b221992229a332c: Sending DeleteTablet for 3 replicas of tablet 2164dc8003cb4fec940dd8bfc03184a1
I20260812 03:08:30.630128 2653 tablet_service.cc:1514] Processing DeleteTablet for tablet 2164dc8003cb4fec940dd8bfc03184a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:30 UTC) from {username='oryx', principal='oryx/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:53988
I20260812 03:08:30.630136 2949 tablet_service.cc:1514] Processing DeleteTablet for tablet 2164dc8003cb4fec940dd8bfc03184a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:30 UTC) from {username='oryx', principal='oryx/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:54334
I20260812 03:08:30.630864 2781 tablet_service.cc:1514] Processing DeleteTablet for tablet 2164dc8003cb4fec940dd8bfc03184a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:30 UTC) from {username='oryx', principal='oryx/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:48316
I20260812 03:08:30.631388 3076 tablet_replica.cc:330] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271: stopping tablet replica
I20260812 03:08:30.631839 3076 raft_consensus.cc:2229] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:08:30.632022 3074 tablet_replica.cc:330] T 2164dc8003cb4fec940dd8bfc03184a1 P f29103f78e19408087681f54d256c8f7: stopping tablet replica
I20260812 03:08:30.632400 3074 raft_consensus.cc:2229] T 2164dc8003cb4fec940dd8bfc03184a1 P f29103f78e19408087681f54d256c8f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:30.632652 3076 raft_consensus.cc:2258] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:30.632802 3074 raft_consensus.cc:2258] T 2164dc8003cb4fec940dd8bfc03184a1 P f29103f78e19408087681f54d256c8f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:30.634224 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 1625
I20260812 03:08:30.634275 3076 ts_tablet_manager.cc:1905] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:30.636122 3074 ts_tablet_manager.cc:1905] T 2164dc8003cb4fec940dd8bfc03184a1 P f29103f78e19408087681f54d256c8f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:30.638522 3076 ts_tablet_manager.cc:1918] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:08:30.638705 3076 log.cc:1196] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/wal/wals/2164dc8003cb4fec940dd8bfc03184a1
I20260812 03:08:30.639138 3076 ts_tablet_manager.cc:1939] T 2164dc8003cb4fec940dd8bfc03184a1 P e2305e11e279480bb5be780cb9570271: Deleting consensus metadata
I20260812 03:08:30.639619 3074 ts_tablet_manager.cc:1918] T 2164dc8003cb4fec940dd8bfc03184a1 P f29103f78e19408087681f54d256c8f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:08:30.639735 3074 log.cc:1196] T 2164dc8003cb4fec940dd8bfc03184a1 P f29103f78e19408087681f54d256c8f7: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-0/wal/wals/2164dc8003cb4fec940dd8bfc03184a1
I20260812 03:08:30.640084 3074 ts_tablet_manager.cc:1939] T 2164dc8003cb4fec940dd8bfc03184a1 P f29103f78e19408087681f54d256c8f7: Deleting consensus metadata
I20260812 03:08:30.642164 3075 tablet_replica.cc:330] T 2164dc8003cb4fec940dd8bfc03184a1 P ac62277b98f34ff58e5d13dee22266bc: stopping tablet replica
I20260812 03:08:30.642606 3075 raft_consensus.cc:2229] T 2164dc8003cb4fec940dd8bfc03184a1 P ac62277b98f34ff58e5d13dee22266bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:30.643033 3075 raft_consensus.cc:2258] T 2164dc8003cb4fec940dd8bfc03184a1 P ac62277b98f34ff58e5d13dee22266bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:30.645061 3075 ts_tablet_manager.cc:1905] T 2164dc8003cb4fec940dd8bfc03184a1 P ac62277b98f34ff58e5d13dee22266bc: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 03:08:30.645579 2603 connection.cc:535] server connection from 127.0.0.1:53988 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 03:08:30.646111 2748 connection.cc:535] server connection from 127.0.0.1:48316 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 03:08:30.648672 3075 ts_tablet_manager.cc:1918] T 2164dc8003cb4fec940dd8bfc03184a1 P ac62277b98f34ff58e5d13dee22266bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:08:30.648803 3075 log.cc:1196] T 2164dc8003cb4fec940dd8bfc03184a1 P ac62277b98f34ff58e5d13dee22266bc: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/2164dc8003cb4fec940dd8bfc03184a1
I20260812 03:08:30.649212 3075 ts_tablet_manager.cc:1939] T 2164dc8003cb4fec940dd8bfc03184a1 P ac62277b98f34ff58e5d13dee22266bc: Deleting consensus metadata
W20260812 03:08:30.650507 2902 connection.cc:441] server connection from 127.0.0.1:54334 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54334 (request call id 1) could send its response
I20260812 03:08:30.654325 2457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.632752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59786 (local address 127.26.250.189:42549)
0812 03:08:30.632999 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:30.633005 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:30.641755 (+ 8750us) server_negotiation.cc:407] Connection header received
0812 03:08:30.641964 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:30.641968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:30.642025 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:30.642112 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:30.643554 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.644398 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.645872 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.646252 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.646687 (+ 435us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.649866 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:30.649871 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:30.649874 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:30.649910 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:30.652113 (+ 2203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:30.652930 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:30.652934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:30.652936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:30.653032 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:30.653351 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:30.653355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:30.653357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:30.653562 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:30.653678 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:30.654021 (+ 343us) server_negotiation.cc:299] Negotiation successful
0812 03:08:30.654159 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178}
I20260812 03:08:30.662245 3037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.651564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40438 (local address 127.26.250.188:39479)
0812 03:08:30.652315 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:30.652319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:30.652334 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:08:30.652380 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:30.652383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:30.652434 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:30.652506 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:30.653424 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.654250 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.655779 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.656240 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.656331 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.658215 (+ 1884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:30.658220 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:30.658223 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:30.658249 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:30.660135 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:30.660676 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:30.660680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:30.660682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:30.660807 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:30.661139 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:30.661145 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:30.661148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:30.661410 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:30.661540 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:30.661913 (+ 373us) server_negotiation.cc:299] Negotiation successful
0812 03:08:30.662051 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":656}
I20260812 03:08:30.663089 3034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.650707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59794 (local address 127.26.250.189:42549)
0812 03:08:30.650823 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:30.650829 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:30.651846 (+ 1017us) server_negotiation.cc:407] Connection header received
0812 03:08:30.652054 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:30.652057 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:30.652113 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:30.652175 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:30.653151 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.653981 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.656280 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.656661 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.657058 (+ 397us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.658734 (+ 1676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:30.658738 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:30.658740 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:30.658766 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:30.660614 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:30.661395 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:30.661398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:30.661400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:30.661484 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:30.661865 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:30.661869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:30.661870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:30.662071 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:30.662166 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:30.662759 (+ 593us) server_negotiation.cc:299] Negotiation successful
0812 03:08:30.662929 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 03:08:30.664693 2730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:30.632878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40428 (local address 127.26.250.188:39479)
0812 03:08:30.632988 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:30.632993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:30.654921 (+ 21928us) server_negotiation.cc:407] Connection header received
0812 03:08:30.655094 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:30.655097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:30.655147 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:30.655197 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:30.656064 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.656857 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.658009 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:30.658450 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:30.658534 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:30.660631 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:30.660637 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:30.660643 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:30.660674 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:30.662441 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:30.663305 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:30.663310 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:30.663313 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:30.663402 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:30.663706 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:30.663710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:30.663712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:30.663957 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:30.664085 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:30.664434 (+ 349us) server_negotiation.cc:299] Negotiation successful
0812 03:08:30.664554 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":30}
I20260812 03:08:31.614952 3085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:31.614422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.26.250.129:45587)
0812 03:08:31.614723 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:31.614853 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":92,"threads_started":1}
W20260812 03:08:31.615357 2723 heartbeater.cc:644] Failed to heartbeat to 127.26.250.190:39831 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
I20260812 03:08:31.616084 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:31.615625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.26.250.130:39823)
0812 03:08:31.615875 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:31.615978 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":67,"threads_started":1}
W20260812 03:08:31.616638 2868 heartbeater.cc:644] Failed to heartbeat to 127.26.250.190:39831 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
I20260812 03:08:31.617609 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:31.617175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.26.250.131:45171)
0812 03:08:31.617454 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:31.617530 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":65,"threads_started":1}
W20260812 03:08:31.617903 3022 heartbeater.cc:644] Failed to heartbeat to 127.26.250.190:39831 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
I20260812 03:08:32.129287 3089 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c767d9b346ba4b9d9b221992229a332c)
I20260812 03:08:32.129549 3089 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: false } }
I20260812 03:08:32.130657 3089 leader_election.cc:288] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c767d9b346ba4b9d9b221992229a332c (127.26.250.190:39831), 137f647ad7304220b2b2fe82f9866715 (127.26.250.189:42549)
I20260812 03:08:32.131211 3091 raft_consensus.cc:482] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c767d9b346ba4b9d9b221992229a332c)
I20260812 03:08:32.131460 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.130620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.0.0.1:57888)
0812 03:08:32.131127 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.131251 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":76,"threads_started":1}
I20260812 03:08:32.131445 3091 raft_consensus.cc:504] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: false } }
W20260812 03:08:32.131840 2512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
I20260812 03:08:32.132817 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.132587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.0.0.1:57892)
0812 03:08:32.132682 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.132735 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260812 03:08:32.133059 2512 leader_election.cc:334] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c767d9b346ba4b9d9b221992229a332c (127.26.250.190:39831): Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
I20260812 03:08:32.133893 3095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.132938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.0.0.1:57904)
0812 03:08:32.133502 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.133720 (+ 218us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":105,"threads_started":1}
I20260812 03:08:32.134020 3091 leader_election.cc:288] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c767d9b346ba4b9d9b221992229a332c (127.26.250.190:39831), 63499cf6e1c048d6be211a8a4fdaafba (127.26.250.188:39479)
W20260812 03:08:32.134320 2303 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
I20260812 03:08:32.136229 3095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.135976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.0.0.1:57906)
0812 03:08:32.136092 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.136161 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 03:08:32.136353 2303 leader_election.cc:334] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c767d9b346ba4b9d9b221992229a332c (127.26.250.190:39831): Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
I20260812 03:08:32.137087 3093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.130946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59798 (local address 127.26.250.189:42549)
0812 03:08:32.131279 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:32.131284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:32.131338 (+ 54us) server_negotiation.cc:407] Connection header received
0812 03:08:32.131532 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:32.131537 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:32.131634 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:32.132410 (+ 776us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:32.132928 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:32.133827 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:32.135691 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:32.136464 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:32.136614 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:32.136683 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:32.136868 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 03:08:32.136955 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":230,"thread_start_us":69,"threads_started":1}
I20260812 03:08:32.137140 3092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.130621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:42549 (local address 127.0.0.1:59798)
0812 03:08:32.131204 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.131222 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:32.131390 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:32.132460 (+ 1070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:32.132495 (+ 35us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:32.132521 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:32.132715 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:32.132751 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:32.133984 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:32.133987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:32.135554 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:32.135565 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:32.136604 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:32.136608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:32.136716 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:32.136741 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 03:08:32.136852 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 03:08:32.136981 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"mutex_wait_us":122,"thread_start_us":106,"threads_started":1}
I20260812 03:08:32.137565 2332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63499cf6e1c048d6be211a8a4fdaafba" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "137f647ad7304220b2b2fe82f9866715" is_pre_election: true
I20260812 03:08:32.137739 2332 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 63499cf6e1c048d6be211a8a4fdaafba in term 1.
I20260812 03:08:32.137966 3096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.133461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:39479 (local address 127.0.0.1:40452)
0812 03:08:32.133648 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.133663 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:32.133799 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:32.134388 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:32.134391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:32.134410 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:32.134504 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:32.134511 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:32.135418 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:32.135421 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:32.136693 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:32.136703 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:32.137524 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:32.137528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:32.137624 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:32.137671 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 03:08:32.137748 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 03:08:32.137832 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":1,"thread_start_us":69,"threads_started":1}
I20260812 03:08:32.137966 3097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.133830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40452 (local address 127.26.250.188:39479)
0812 03:08:32.134118 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:32.134130 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:32.134147 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:08:32.134191 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:32.134194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:32.134239 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:32.134641 (+ 402us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:32.134745 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:32.135253 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:32.136820 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:32.137409 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:32.137524 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:32.137570 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:32.137764 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 03:08:32.137835 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":207,"thread_start_us":119,"threads_started":1}
I20260812 03:08:32.138092 2514 leader_election.cc:302] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 137f647ad7304220b2b2fe82f9866715, 63499cf6e1c048d6be211a8a4fdaafba; no voters: c767d9b346ba4b9d9b221992229a332c
I20260812 03:08:32.138331 3089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 03:08:32.138478 3089 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 1 FOLLOWER]: Starting leader election (detected failure of leader c767d9b346ba4b9d9b221992229a332c)
I20260812 03:08:32.138536 3089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:08:32.138687 2540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "137f647ad7304220b2b2fe82f9866715" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "63499cf6e1c048d6be211a8a4fdaafba" is_pre_election: true
I20260812 03:08:32.139688 3089 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: false } }
I20260812 03:08:32.140002 2540 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 137f647ad7304220b2b2fe82f9866715 in current term 2: Already voted for candidate 63499cf6e1c048d6be211a8a4fdaafba in this term.
I20260812 03:08:32.140069 3089 leader_election.cc:288] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [CANDIDATE]: Term 2 election: Requested vote from peers c767d9b346ba4b9d9b221992229a332c (127.26.250.190:39831), 137f647ad7304220b2b2fe82f9866715 (127.26.250.189:42549)
I20260812 03:08:32.140447 3092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.140228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.0.0.1:57914)
0812 03:08:32.140323 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.140368 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 03:08:32.140799 2303 leader_election.cc:302] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 137f647ad7304220b2b2fe82f9866715; no voters: 63499cf6e1c048d6be211a8a4fdaafba, c767d9b346ba4b9d9b221992229a332c
I20260812 03:08:32.140854 2332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "63499cf6e1c048d6be211a8a4fdaafba" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "137f647ad7304220b2b2fe82f9866715"
I20260812 03:08:32.141000 2332 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:08:32.141330 3092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.141115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.0.0.1:57922)
0812 03:08:32.141222 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.141280 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 03:08:32.141508 2512 leader_election.cc:334] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c767d9b346ba4b9d9b221992229a332c (127.26.250.190:39831): Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
I20260812 03:08:32.142632 2332 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 63499cf6e1c048d6be211a8a4fdaafba in term 2.
I20260812 03:08:32.142767 3091 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:08:32.143008 2514 leader_election.cc:302] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 137f647ad7304220b2b2fe82f9866715, 63499cf6e1c048d6be211a8a4fdaafba; no voters: c767d9b346ba4b9d9b221992229a332c
I20260812 03:08:32.143265 3089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:08:32.143527 3089 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [term 2 LEADER]: Becoming Leader. State: Replica: 63499cf6e1c048d6be211a8a4fdaafba, State: Running, Role: LEADER
I20260812 03:08:32.143811 3089 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: false } }
I20260812 03:08:32.144970 3100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63499cf6e1c048d6be211a8a4fdaafba. Latest consensus state: current_term: 2 leader_uuid: "63499cf6e1c048d6be211a8a4fdaafba" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: false } } }
I20260812 03:08:32.145097 3100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:32.145440 3102 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:08:32.148191 3102 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:08:32.148641 3102 catalog_manager.cc:1259] Loaded cluster ID: 762bd694924c4023850f1473ebe1b5bd
I20260812 03:08:32.148694 3102 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:08:32.149327 3102 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:08:32.149792 3102 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba: Loaded TSK: 0
I20260812 03:08:32.150660 3102 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:08:32.200376 2529 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40438:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:08:32.203655 2529 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:08:32.206784 3092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.206522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.0.0.1:57956)
0812 03:08:32.206646 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.206710 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 03:08:32.206876 2332 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [term 2 FOLLOWER]: Refusing update from remote peer 63499cf6e1c048d6be211a8a4fdaafba: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260812 03:08:32.207484 3100 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [LEADER]: Connected to new peer: Peer: permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260812 03:08:32.207592 3092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.207369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.0.0.1:57960)
0812 03:08:32.207481 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.207536 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 03:08:32.207707 2512 consensus_peers.cc:487] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba -> Peer c767d9b346ba4b9d9b221992229a332c (127.26.250.190:39831): Couldn't send request to peer c767d9b346ba4b9d9b221992229a332c. Status: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 03:08:32.209152 3091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63499cf6e1c048d6be211a8a4fdaafba. Latest consensus state: current_term: 2 leader_uuid: "63499cf6e1c048d6be211a8a4fdaafba" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: false } } }
I20260812 03:08:32.209307 3091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:08:32.210702 3089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "63499cf6e1c048d6be211a8a4fdaafba" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: false } } }
I20260812 03:08:32.210865 3089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:32.211308 3091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "63499cf6e1c048d6be211a8a4fdaafba" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c767d9b346ba4b9d9b221992229a332c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39831 } } peers { permanent_uuid: "137f647ad7304220b2b2fe82f9866715" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42549 } attrs { promote: false } } peers { permanent_uuid: "63499cf6e1c048d6be211a8a4fdaafba" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39479 } attrs { promote: false } } }
I20260812 03:08:32.211431 3091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 137f647ad7304220b2b2fe82f9866715 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:08:32.222472 3092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.217773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:39537 (local address 127.0.0.1:54348)
0812 03:08:32.217889 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.217905 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:32.217986 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:32.218626 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:32.218649 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:32.218670 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:32.218780 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:32.218786 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:32.219830 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:32.219833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:32.221124 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:32.221135 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:32.222031 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:32.222035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:32.222125 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:32.222147 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:08:32.222220 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 03:08:32.222298 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":100}
I20260812 03:08:32.222476 3109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.217915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54348 (local address 127.26.250.131:39537)
0812 03:08:32.218238 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:32.218243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:32.218259 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:08:32.218430 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:32.218435 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:32.218492 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:32.218616 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:32.218926 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:32.219700 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:32.221264 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:32.221914 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:32.222036 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:32.222096 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:32.222245 (+ 149us) server_negotiation.cc:299] Negotiation successful
0812 03:08:32.222327 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":118,"threads_started":1}
I20260812 03:08:32.222766 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.218274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:36363 (local address 127.0.0.1:48332)
0812 03:08:32.218385 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.218402 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:32.218490 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:32.219061 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:32.219064 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:32.219087 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:32.219189 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:32.219197 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:32.220318 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:32.220324 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:32.221482 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:32.221491 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:32.222376 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:32.222380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:32.222473 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:32.222501 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 03:08:32.222552 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 03:08:32.222634 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":93}
I20260812 03:08:32.223049 2949 tablet_service.cc:1467] Processing CreateTablet for tablet 2fd3b403bae248fea85ebadd9328b884 (DEFAULT_TABLE table=test-table [id=9a4bc425b33e4f42bb873f92929e1736]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:32.223152 3112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.218788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:46261 (local address 127.0.0.1:54000)
0812 03:08:32.219052 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.219064 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:32.219131 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:32.219389 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:32.219392 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:32.219408 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:32.219507 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:32.219513 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:32.220510 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:32.220516 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:32.221711 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:32.221721 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:32.222715 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:32.222718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:32.222818 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:32.222844 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 03:08:32.222949 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 03:08:32.223018 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":26,"thread_start_us":110,"threads_started":1}
I20260812 03:08:32.223306 3110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.218564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48332 (local address 127.26.250.130:36363)
0812 03:08:32.218809 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:32.218816 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:32.218831 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:08:32.218886 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:32.218891 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:32.218939 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:32.219294 (+ 355us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:32.219447 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:32.220184 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:32.221602 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:32.222200 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:32.222979 (+ 779us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:32.223031 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:32.223067 (+ 36us) server_negotiation.cc:299] Negotiation successful
0812 03:08:32.223145 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":79,"threads_started":1}
I20260812 03:08:32.223399 2949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fd3b403bae248fea85ebadd9328b884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:32.223707 3111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.218552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54000 (local address 127.26.250.129:46261)
0812 03:08:32.219132 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:32.219137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:32.219152 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:08:32.219202 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:32.219205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:32.219248 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:32.219338 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:32.219624 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:32.220415 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:32.221840 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:32.222476 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:32.223362 (+ 886us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:32.223423 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:32.223460 (+ 37us) server_negotiation.cc:299] Negotiation successful
0812 03:08:32.223567 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":103,"threads_started":1}
I20260812 03:08:32.224203 2653 tablet_service.cc:1467] Processing CreateTablet for tablet 2fd3b403bae248fea85ebadd9328b884 (DEFAULT_TABLE table=test-table [id=9a4bc425b33e4f42bb873f92929e1736]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:32.224483 2653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fd3b403bae248fea85ebadd9328b884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:32.225798 3113 tablet_bootstrap.cc:492] T 2fd3b403bae248fea85ebadd9328b884 P ac62277b98f34ff58e5d13dee22266bc: Bootstrap starting.
I20260812 03:08:32.226397 2781 tablet_service.cc:1467] Processing CreateTablet for tablet 2fd3b403bae248fea85ebadd9328b884 (DEFAULT_TABLE table=test-table [id=9a4bc425b33e4f42bb873f92929e1736]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:32.226676 2781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fd3b403bae248fea85ebadd9328b884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:32.226951 3113 tablet_bootstrap.cc:654] T 2fd3b403bae248fea85ebadd9328b884 P ac62277b98f34ff58e5d13dee22266bc: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:32.228364 3113 tablet_bootstrap.cc:492] T 2fd3b403bae248fea85ebadd9328b884 P ac62277b98f34ff58e5d13dee22266bc: No bootstrap required, opened a new log
I20260812 03:08:32.228458 3113 ts_tablet_manager.cc:1397] T 2fd3b403bae248fea85ebadd9328b884 P ac62277b98f34ff58e5d13dee22266bc: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 03:08:32.228960 3113 raft_consensus.cc:348] T 2fd3b403bae248fea85ebadd9328b884 P ac62277b98f34ff58e5d13dee22266bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac62277b98f34ff58e5d13dee22266bc" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39537 } } peers { permanent_uuid: "e2305e11e279480bb5be780cb9570271" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 36363 } } peers { permanent_uuid: "f29103f78e19408087681f54d256c8f7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46261 } }
I20260812 03:08:32.229100 3113 raft_consensus.cc:374] T 2fd3b403bae248fea85ebadd9328b884 P ac62277b98f34ff58e5d13dee22266bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:32.229135 3113 raft_consensus.cc:729] T 2fd3b403bae248fea85ebadd9328b884 P ac62277b98f34ff58e5d13dee22266bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac62277b98f34ff58e5d13dee22266bc, State: Initialized, Role: FOLLOWER
I20260812 03:08:32.229274 3116 tablet_bootstrap.cc:492] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271: Bootstrap starting.
I20260812 03:08:32.229257 3113 consensus_queue.cc:260] T 2fd3b403bae248fea85ebadd9328b884 P ac62277b98f34ff58e5d13dee22266bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac62277b98f34ff58e5d13dee22266bc" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39537 } } peers { permanent_uuid: "e2305e11e279480bb5be780cb9570271" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 36363 } } peers { permanent_uuid: "f29103f78e19408087681f54d256c8f7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46261 } }
I20260812 03:08:32.229882 3114 tablet_bootstrap.cc:492] T 2fd3b403bae248fea85ebadd9328b884 P f29103f78e19408087681f54d256c8f7: Bootstrap starting.
I20260812 03:08:32.230394 3116 tablet_bootstrap.cc:654] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:32.230542 3118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.229972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.26.250.131:40867)
0812 03:08:32.230246 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.230331 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":100,"threads_started":1}
I20260812 03:08:32.230861 3114 tablet_bootstrap.cc:654] T 2fd3b403bae248fea85ebadd9328b884 P f29103f78e19408087681f54d256c8f7: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:32.231930 3116 tablet_bootstrap.cc:492] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271: No bootstrap required, opened a new log
I20260812 03:08:32.232028 3116 ts_tablet_manager.cc:1397] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 03:08:32.232381 3114 tablet_bootstrap.cc:492] T 2fd3b403bae248fea85ebadd9328b884 P f29103f78e19408087681f54d256c8f7: No bootstrap required, opened a new log
I20260812 03:08:32.232481 3114 ts_tablet_manager.cc:1397] T 2fd3b403bae248fea85ebadd9328b884 P f29103f78e19408087681f54d256c8f7: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 03:08:32.232564 3116 raft_consensus.cc:348] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac62277b98f34ff58e5d13dee22266bc" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39537 } } peers { permanent_uuid: "e2305e11e279480bb5be780cb9570271" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 36363 } } peers { permanent_uuid: "f29103f78e19408087681f54d256c8f7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46261 } }
I20260812 03:08:32.232688 3116 raft_consensus.cc:374] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:32.232748 3116 raft_consensus.cc:729] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2305e11e279480bb5be780cb9570271, State: Initialized, Role: FOLLOWER
I20260812 03:08:32.232903 3116 consensus_queue.cc:260] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac62277b98f34ff58e5d13dee22266bc" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39537 } } peers { permanent_uuid: "e2305e11e279480bb5be780cb9570271" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 36363 } } peers { permanent_uuid: "f29103f78e19408087681f54d256c8f7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46261 } }
I20260812 03:08:32.232945 3114 raft_consensus.cc:348] T 2fd3b403bae248fea85ebadd9328b884 P f29103f78e19408087681f54d256c8f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac62277b98f34ff58e5d13dee22266bc" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39537 } } peers { permanent_uuid: "e2305e11e279480bb5be780cb9570271" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 36363 } } peers { permanent_uuid: "f29103f78e19408087681f54d256c8f7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46261 } }
I20260812 03:08:32.233057 3114 raft_consensus.cc:374] T 2fd3b403bae248fea85ebadd9328b884 P f29103f78e19408087681f54d256c8f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:32.233089 3114 raft_consensus.cc:729] T 2fd3b403bae248fea85ebadd9328b884 P f29103f78e19408087681f54d256c8f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f29103f78e19408087681f54d256c8f7, State: Initialized, Role: FOLLOWER
I20260812 03:08:32.233268 3116 ts_tablet_manager.cc:1428] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:08:32.233232 3114 consensus_queue.cc:260] T 2fd3b403bae248fea85ebadd9328b884 P f29103f78e19408087681f54d256c8f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac62277b98f34ff58e5d13dee22266bc" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39537 } } peers { permanent_uuid: "e2305e11e279480bb5be780cb9570271" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 36363 } } peers { permanent_uuid: "f29103f78e19408087681f54d256c8f7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46261 } }
I20260812 03:08:32.233589 3114 ts_tablet_manager.cc:1428] T 2fd3b403bae248fea85ebadd9328b884 P f29103f78e19408087681f54d256c8f7: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260812 03:08:32.234915 3113 ts_tablet_manager.cc:1428] T 2fd3b403bae248fea85ebadd9328b884 P ac62277b98f34ff58e5d13dee22266bc: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260812 03:08:32.235087 3020 heartbeater.cc:503] Master 127.26.250.188:39479 was elected leader, sending a full tablet report...
I20260812 03:08:32.235128 2867 heartbeater.cc:503] Master 127.26.250.188:39479 was elected leader, sending a full tablet report...
I20260812 03:08:32.235163 3124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.234285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.26.250.130:42833)
0812 03:08:32.234987 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.235059 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":118,"threads_started":1}
I20260812 03:08:32.236591 3118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.236244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.26.250.131:41125)
0812 03:08:32.236425 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.236484 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260812 03:08:32.237200 3124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.235641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.26.250.130:44223)
0812 03:08:32.237029 (+ 1388us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.237092 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1320,"mutex_wait_us":1045}
I20260812 03:08:32.238056 3123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.234017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.26.250.129:50801)
0812 03:08:32.237907 (+ 3890us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.237957 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3831,"thread_start_us":96,"threads_started":1}
I20260812 03:08:32.238440 2722 heartbeater.cc:503] Master 127.26.250.188:39479 was elected leader, sending a full tablet report...
I20260812 03:08:32.238900 3123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.238602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.26.250.129:54339)
0812 03:08:32.238729 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.238825 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 03:08:32.351096 3121 raft_consensus.cc:482] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:08:32.351362 3121 raft_consensus.cc:504] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac62277b98f34ff58e5d13dee22266bc" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39537 } } peers { permanent_uuid: "e2305e11e279480bb5be780cb9570271" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 36363 } } peers { permanent_uuid: "f29103f78e19408087681f54d256c8f7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46261 } }
I20260812 03:08:32.351904 3121 leader_election.cc:288] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ac62277b98f34ff58e5d13dee22266bc (127.26.250.131:39537), f29103f78e19408087681f54d256c8f7 (127.26.250.129:46261)
I20260812 03:08:32.352568 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fd3b403bae248fea85ebadd9328b884" candidate_uuid: "e2305e11e279480bb5be780cb9570271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f29103f78e19408087681f54d256c8f7" is_pre_election: true
I20260812 03:08:32.352579 2970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fd3b403bae248fea85ebadd9328b884" candidate_uuid: "e2305e11e279480bb5be780cb9570271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac62277b98f34ff58e5d13dee22266bc" is_pre_election: true
I20260812 03:08:32.352770 2673 raft_consensus.cc:2454] T 2fd3b403bae248fea85ebadd9328b884 P f29103f78e19408087681f54d256c8f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2305e11e279480bb5be780cb9570271 in term 0.
I20260812 03:08:32.352794 2970 raft_consensus.cc:2454] T 2fd3b403bae248fea85ebadd9328b884 P ac62277b98f34ff58e5d13dee22266bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2305e11e279480bb5be780cb9570271 in term 0.
I20260812 03:08:32.353205 2749 leader_election.cc:302] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac62277b98f34ff58e5d13dee22266bc, e2305e11e279480bb5be780cb9570271; no voters:
I20260812 03:08:32.353466 3121 raft_consensus.cc:2781] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:08:32.353549 3121 raft_consensus.cc:482] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:08:32.353582 3121 raft_consensus.cc:3037] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:32.354916 3121 raft_consensus.cc:504] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac62277b98f34ff58e5d13dee22266bc" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39537 } } peers { permanent_uuid: "e2305e11e279480bb5be780cb9570271" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 36363 } } peers { permanent_uuid: "f29103f78e19408087681f54d256c8f7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46261 } }
I20260812 03:08:32.355365 3121 leader_election.cc:288] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271 [CANDIDATE]: Term 1 election: Requested vote from peers ac62277b98f34ff58e5d13dee22266bc (127.26.250.131:39537), f29103f78e19408087681f54d256c8f7 (127.26.250.129:46261)
I20260812 03:08:32.355846 2970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fd3b403bae248fea85ebadd9328b884" candidate_uuid: "e2305e11e279480bb5be780cb9570271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac62277b98f34ff58e5d13dee22266bc"
I20260812 03:08:32.356000 2970 raft_consensus.cc:3037] T 2fd3b403bae248fea85ebadd9328b884 P ac62277b98f34ff58e5d13dee22266bc [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:32.356003 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fd3b403bae248fea85ebadd9328b884" candidate_uuid: "e2305e11e279480bb5be780cb9570271" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f29103f78e19408087681f54d256c8f7"
I20260812 03:08:32.356165 2673 raft_consensus.cc:3037] T 2fd3b403bae248fea85ebadd9328b884 P f29103f78e19408087681f54d256c8f7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:32.357077 2970 raft_consensus.cc:2454] T 2fd3b403bae248fea85ebadd9328b884 P ac62277b98f34ff58e5d13dee22266bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2305e11e279480bb5be780cb9570271 in term 1.
I20260812 03:08:32.357254 2673 raft_consensus.cc:2454] T 2fd3b403bae248fea85ebadd9328b884 P f29103f78e19408087681f54d256c8f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2305e11e279480bb5be780cb9570271 in term 1.
I20260812 03:08:32.357450 2749 leader_election.cc:302] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac62277b98f34ff58e5d13dee22266bc, e2305e11e279480bb5be780cb9570271; no voters:
I20260812 03:08:32.357646 3121 raft_consensus.cc:2781] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:32.357899 3121 raft_consensus.cc:686] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271 [term 1 LEADER]: Becoming Leader. State: Replica: e2305e11e279480bb5be780cb9570271, State: Running, Role: LEADER
I20260812 03:08:32.358083 3121 consensus_queue.cc:237] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac62277b98f34ff58e5d13dee22266bc" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39537 } } peers { permanent_uuid: "e2305e11e279480bb5be780cb9570271" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 36363 } } peers { permanent_uuid: "f29103f78e19408087681f54d256c8f7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46261 } }
I20260812 03:08:32.358810 3124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.358499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.26.250.130:47959)
0812 03:08:32.358671 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.358736 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260812 03:08:32.360122 2529 catalog_manager.cc:5495] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2305e11e279480bb5be780cb9570271 (127.26.250.130). New cstate: current_term: 1 leader_uuid: "e2305e11e279480bb5be780cb9570271" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac62277b98f34ff58e5d13dee22266bc" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2305e11e279480bb5be780cb9570271" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 36363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f29103f78e19408087681f54d256c8f7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46261 } health_report { overall_health: UNKNOWN } } }
I20260812 03:08:32.379565 3110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.376256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48346 (local address 127.26.250.130:36363)
0812 03:08:32.376401 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:32.376405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:32.376418 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:08:32.376465 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:32.376467 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:32.376517 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:32.376582 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:08:32.376894 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:32.377403 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:32.378275 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:32.378596 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:32.378680 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:32.378923 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:08:32.378992 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:08:32.379059 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:32.379276 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 03:08:32.379372 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 03:08:32.381999 2673 raft_consensus.cc:1261] T 2fd3b403bae248fea85ebadd9328b884 P f29103f78e19408087681f54d256c8f7 [term 1 FOLLOWER]: Refusing update from remote peer e2305e11e279480bb5be780cb9570271: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:32.382033 2970 raft_consensus.cc:1261] T 2fd3b403bae248fea85ebadd9328b884 P ac62277b98f34ff58e5d13dee22266bc [term 1 FOLLOWER]: Refusing update from remote peer e2305e11e279480bb5be780cb9570271: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:32.382547 3125 consensus_queue.cc:1035] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac62277b98f34ff58e5d13dee22266bc" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:32.382766 3121 consensus_queue.cc:1035] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f29103f78e19408087681f54d256c8f7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:32.384495 3118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.384122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.26.250.131:35915)
0812 03:08:32.384313 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.384392 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260812 03:08:32.386945 3123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.384973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.26.250.129:47087)
0812 03:08:32.386780 (+ 1807us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.386836 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1096}
I20260812 03:08:32.388957 3124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.388700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.26.250.130:52869)
0812 03:08:32.388805 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.388865 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260812 03:08:32.389321 3123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.388241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.26.250.129:36199)
0812 03:08:32.389187 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.389246 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":874}
I20260812 03:08:32.389742 3129 mvcc.cc:205] Tried to move back new op lower bound from 7317520844312174592 to 7317520844220588032. Current Snapshot: MvccSnapshot[applied={T|T < 7317520844312174592 or (T in {7317520844312174592})}]
I20260812 03:08:32.391294 3124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.390993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.26.250.130:53707)
0812 03:08:32.391151 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.391224 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260812 03:08:32.392673 3118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.392322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39831 (local address 127.26.250.131:39341)
0812 03:08:32.392522 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.392576 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:39831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260812 03:08:32.393760 2529 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:32.394026 2529 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:32.398370 2529 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63499cf6e1c048d6be211a8a4fdaafba: Sending DeleteTablet for 3 replicas of tablet 2fd3b403bae248fea85ebadd9328b884
I20260812 03:08:32.399216 2949 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fd3b403bae248fea85ebadd9328b884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:32 UTC) from {username='oryx', principal='oryx/127.26.250.188@KRBTEST.COM'} at 127.0.0.1:54348
I20260812 03:08:32.399231 2781 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fd3b403bae248fea85ebadd9328b884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:32 UTC) from {username='oryx', principal='oryx/127.26.250.188@KRBTEST.COM'} at 127.0.0.1:48332
I20260812 03:08:32.399408 2653 tablet_service.cc:1514] Processing DeleteTablet for tablet 2fd3b403bae248fea85ebadd9328b884 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:32 UTC) from {username='oryx', principal='oryx/127.26.250.188@KRBTEST.COM'} at 127.0.0.1:54000
I20260812 03:08:32.399765 3140 tablet_replica.cc:330] T 2fd3b403bae248fea85ebadd9328b884 P ac62277b98f34ff58e5d13dee22266bc: stopping tablet replica
I20260812 03:08:32.399783 3142 tablet_replica.cc:330] T 2fd3b403bae248fea85ebadd9328b884 P f29103f78e19408087681f54d256c8f7: stopping tablet replica
I20260812 03:08:32.399912 3140 raft_consensus.cc:2229] T 2fd3b403bae248fea85ebadd9328b884 P ac62277b98f34ff58e5d13dee22266bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:32.399921 3142 raft_consensus.cc:2229] T 2fd3b403bae248fea85ebadd9328b884 P f29103f78e19408087681f54d256c8f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:32.400072 3140 raft_consensus.cc:2258] T 2fd3b403bae248fea85ebadd9328b884 P ac62277b98f34ff58e5d13dee22266bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:32.400074 3142 raft_consensus.cc:2258] T 2fd3b403bae248fea85ebadd9328b884 P f29103f78e19408087681f54d256c8f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:32.400126 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 2479
I20260812 03:08:32.400360 3141 tablet_replica.cc:330] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271: stopping tablet replica
I20260812 03:08:32.400513 3141 raft_consensus.cc:2229] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:08:32.400715 3141 raft_consensus.cc:2258] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:32.402123 3141 ts_tablet_manager.cc:1905] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:32.406280 3141 ts_tablet_manager.cc:1918] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:08:32.406422 3141 log.cc:1196] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-1/wal/wals/2fd3b403bae248fea85ebadd9328b884
I20260812 03:08:32.406965 3141 ts_tablet_manager.cc:1939] T 2fd3b403bae248fea85ebadd9328b884 P e2305e11e279480bb5be780cb9570271: Deleting consensus metadata
I20260812 03:08:32.407892 2511 catalog_manager.cc:4841] TS e2305e11e279480bb5be780cb9570271 (127.26.250.130:36363): tablet 2fd3b403bae248fea85ebadd9328b884 (table test-table [id=9a4bc425b33e4f42bb873f92929e1736]) successfully deleted
I20260812 03:08:32.411125 3140 ts_tablet_manager.cc:1905] T 2fd3b403bae248fea85ebadd9328b884 P ac62277b98f34ff58e5d13dee22266bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:32.413390 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 2731
I20260812 03:08:32.414994 3140 ts_tablet_manager.cc:1918] T 2fd3b403bae248fea85ebadd9328b884 P ac62277b98f34ff58e5d13dee22266bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:08:32.415098 3140 log.cc:1196] T 2fd3b403bae248fea85ebadd9328b884 P ac62277b98f34ff58e5d13dee22266bc: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/2fd3b403bae248fea85ebadd9328b884
I20260812 03:08:32.415107 3112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:32.413619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:46261 (local address 127.0.0.1:54008)
0812 03:08:32.414920 (+ 1301us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:32.414992 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.129:46261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1183}
I20260812 03:08:32.415412 3140 ts_tablet_manager.cc:1939] T 2fd3b403bae248fea85ebadd9328b884 P ac62277b98f34ff58e5d13dee22266bc: Deleting consensus metadata
W20260812 03:08:32.415529 2512 catalog_manager.cc:4584] TS f29103f78e19408087681f54d256c8f7 (127.26.250.129:46261): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2fd3b403bae248fea85ebadd9328b884: Network error: Client connection negotiation failed: client connection to 127.26.250.129:46261: connect: Connection refused (error 111)
I20260812 03:08:32.416356 2512 catalog_manager.cc:4841] TS ac62277b98f34ff58e5d13dee22266bc (127.26.250.131:39537): tablet 2fd3b403bae248fea85ebadd9328b884 (table test-table [id=9a4bc425b33e4f42bb873f92929e1736]) successfully deleted
I20260812 03:08:32.423566 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 2874
I20260812 03:08:32.432065 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 2172
I20260812 03:08:32.441536 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 2281
2026-08-12T03:08:32Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5721 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:08:32 dist-test-slave-zpfg krb5kdc[3145](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:08:32 dist-test-slave-zpfg krb5kdc[3145](info): set up 2 sockets
Aug 12 03:08:32 dist-test-slave-zpfg krb5kdc[3145](info): commencing operation
krb5kdc: starting...
W20260812 03:08:33.609684 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:08:33 dist-test-slave-zpfg krb5kdc[3145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:08:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:08:33Z Disabled control of system clock
WARNING: no policy specified for oryx/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.190.keytab.
Entry for principal oryx/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.190.keytab.
I20260812 03:08:33.758740 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:32913
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.190.keytab
--principal=oryx/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:44285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:32913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:33.884128 3159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:33.884513 3159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:33.884613 3159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:33.889462 3159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:08:33.889578 3159 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 03:08:33.889614 3159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:33.889665 3159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:33.889724 3159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:08:33.889765 3159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:08:33.896061 3159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:44285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:32913
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.190.keytab
--principal=oryx/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:32913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:33.897418 3159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:33.898736 3159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:33.908756 3164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:33.908802 3165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:33.908803 3167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:33.908936 3159 server_base.cc:1034] running on GCE node
I20260812 03:08:33.909718 3159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:33.911125 3159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:33.912341 3159 hybrid_clock.cc:648] HybridClock initialized: now 1786504113912312 us; error 47 us; skew 500 ppm
Aug 12 03:08:33 dist-test-slave-zpfg krb5kdc[3145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504113, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:33.923612 3159 init.cc:377] Logged in from keytab as oryx/127.26.250.190@KRBTEST.COM (short username oryx)
I20260812 03:08:33.924811 3159 webserver.cc:460] Webserver started at http://127.26.250.190:34721/ using document root <none> and password file <none>
I20260812 03:08:33.925455 3159 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:33.925530 3159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:33.925730 3159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:33.928217 3159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "712dad23c9c343f29dbc9bc3fb5d04cf"
format_stamp: "Formatted at 2026-08-12 03:08:33 on dist-test-slave-zpfg"
server_key: "ffa67a64faf2cfd0ab1d316c2f079cc2"
server_key_iv: "b4eb81f4a0ca94bf121b14bf85dd5158"
server_key_version: "encryptionkey@0"
I20260812 03:08:33.928851 3159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "712dad23c9c343f29dbc9bc3fb5d04cf"
format_stamp: "Formatted at 2026-08-12 03:08:33 on dist-test-slave-zpfg"
server_key: "ffa67a64faf2cfd0ab1d316c2f079cc2"
server_key_iv: "b4eb81f4a0ca94bf121b14bf85dd5158"
server_key_version: "encryptionkey@0"
I20260812 03:08:33.933467 3159 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 03:08:33.936556 3174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:33.938038 3159 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:08:33.938199 3159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "712dad23c9c343f29dbc9bc3fb5d04cf"
format_stamp: "Formatted at 2026-08-12 03:08:33 on dist-test-slave-zpfg"
server_key: "ffa67a64faf2cfd0ab1d316c2f079cc2"
server_key_iv: "b4eb81f4a0ca94bf121b14bf85dd5158"
server_key_version: "encryptionkey@0"
I20260812 03:08:33.938405 3159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:33.956419 3159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:33.960427 3159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:33.960669 3159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:33.970515 3159 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:32913
I20260812 03:08:33.970544 3239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:32913 every 8 connection(s)
I20260812 03:08:33.971644 3159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:08:33.974557 3240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:33.977588 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 3159
I20260812 03:08:33.977736 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:08:33.978080 27626 external_mini_cluster.cc:1428] Setting key d58c504ed0d8e5fa81371b46052db6e8
I20260812 03:08:33.981262 3240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf: Bootstrap starting.
I20260812 03:08:33.983677 3240 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:33.984465 3240 log.cc:824] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:33.986673 3240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf: No bootstrap required, opened a new log
Aug 12 03:08:33 dist-test-slave-zpfg krb5kdc[3145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM
I20260812 03:08:33.990538 3240 raft_consensus.cc:348] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "712dad23c9c343f29dbc9bc3fb5d04cf" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 32913 } }
I20260812 03:08:33.990796 3240 raft_consensus.cc:374] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:33.990882 3240 raft_consensus.cc:729] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 712dad23c9c343f29dbc9bc3fb5d04cf, State: Initialized, Role: FOLLOWER
I20260812 03:08:33.991350 3240 consensus_queue.cc:260] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "712dad23c9c343f29dbc9bc3fb5d04cf" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 32913 } }
I20260812 03:08:33.991506 3240 raft_consensus.cc:388] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:08:33.991596 3240 raft_consensus.cc:482] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:08:33.991704 3240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:33.992944 3240 raft_consensus.cc:504] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "712dad23c9c343f29dbc9bc3fb5d04cf" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 32913 } }
I20260812 03:08:33.993319 3240 leader_election.cc:302] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 712dad23c9c343f29dbc9bc3fb5d04cf; no voters:
I20260812 03:08:33.993683 3240 leader_election.cc:288] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:08:33.993901 3246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:33.994174 3246 raft_consensus.cc:686] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf [term 1 LEADER]: Becoming Leader. State: Replica: 712dad23c9c343f29dbc9bc3fb5d04cf, State: Running, Role: LEADER
I20260812 03:08:33.994179 3243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:33.979750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47304 (local address 127.26.250.190:32913)
0812 03:08:33.980280 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:33.980297 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:33.980337 (+ 40us) server_negotiation.cc:407] Connection header received
0812 03:08:33.980993 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:33.981011 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:33.981425 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:33.981835 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:33.982757 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:33.983844 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:33.984744 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:33.985469 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:33.985649 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:33.988374 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:33.988380 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:33.988393 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:33.988420 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:33.990478 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:33.991020 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:33.991027 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:33.991041 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:33.991212 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:33.991625 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:33.991629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:33.991632 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:33.992200 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:33.992412 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:33.992750 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 03:08:33.993107 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":153,"threads_started":1}
I20260812 03:08:33.994586 3246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "712dad23c9c343f29dbc9bc3fb5d04cf" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 32913 } }
I20260812 03:08:33.995033 3240 sys_catalog.cc:564] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:08:33.997381 3248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 712dad23c9c343f29dbc9bc3fb5d04cf. Latest consensus state: current_term: 1 leader_uuid: "712dad23c9c343f29dbc9bc3fb5d04cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "712dad23c9c343f29dbc9bc3fb5d04cf" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 32913 } } }
I20260812 03:08:33.997653 3247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "712dad23c9c343f29dbc9bc3fb5d04cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "712dad23c9c343f29dbc9bc3fb5d04cf" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 32913 } } }
I20260812 03:08:33.997743 3248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:33.997766 3247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf [sys.catalog]: This master's current role is: LEADER
W20260812 03:08:34.000767 3261 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:08:34.000932 3261 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:08:34.001034 3263 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:08:34.001999 3263 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:08:34.009585 3263 catalog_manager.cc:1347] Generated new cluster ID: fad310dd053544f099bcfb1dae5715fc
I20260812 03:08:34.009701 3263 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:08:34.021544 3263 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:08:34.022962 3263 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:08:34.032068 3263 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf: Generated new TSK 0
I20260812 03:08:34.033022 3263 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.26.250.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.250.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.250.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.129.keytab.
Entry for principal oryx/127.26.250.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.129.keytab.
I20260812 03:08:34.074105 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:0
--local_ip_for_outbound_sockets=127.26.250.129
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:32913
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.129.keytab
--principal=oryx/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:44285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:34.209849 3268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:34.210253 3268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:34.210362 3268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:34.215050 3268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:34.215183 3268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:34.215265 3268 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:08:34.215355 3268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:08:34.221645 3268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:44285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.129.keytab
--principal=oryx/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.250.190:32913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:34.222995 3268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:34.224190 3268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:34.236020 3276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:34.235939 3273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:34.236114 3268 server_base.cc:1034] running on GCE node
W20260812 03:08:34.236052 3274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:34.237051 3268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:34.237869 3268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:34.239071 3268 hybrid_clock.cc:648] HybridClock initialized: now 1786504114239039 us; error 50 us; skew 500 ppm
Aug 12 03:08:34 dist-test-slave-zpfg krb5kdc[3145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504114, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:34.242182 3268 init.cc:377] Logged in from keytab as oryx/127.26.250.129@KRBTEST.COM (short username oryx)
I20260812 03:08:34.243386 3268 webserver.cc:460] Webserver started at http://127.26.250.129:41157/ using document root <none> and password file <none>
I20260812 03:08:34.244020 3268 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:34.244124 3268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:34.244346 3268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:34.246623 3268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-0/data/instance:
uuid: "6801eb5071dc43bcbd45983831f97ce7"
format_stamp: "Formatted at 2026-08-12 03:08:34 on dist-test-slave-zpfg"
server_key: "6e85ea294acd85701523fe9f686afef7"
server_key_iv: "ad19a84d8e0ff789002e8138dd88ebf3"
server_key_version: "encryptionkey@0"
I20260812 03:08:34.247228 3268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance:
uuid: "6801eb5071dc43bcbd45983831f97ce7"
format_stamp: "Formatted at 2026-08-12 03:08:34 on dist-test-slave-zpfg"
server_key: "6e85ea294acd85701523fe9f686afef7"
server_key_iv: "ad19a84d8e0ff789002e8138dd88ebf3"
server_key_version: "encryptionkey@0"
I20260812 03:08:34.251296 3268 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:08:34.254491 3285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:34.255894 3268 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:08:34.256055 3268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "6801eb5071dc43bcbd45983831f97ce7"
format_stamp: "Formatted at 2026-08-12 03:08:34 on dist-test-slave-zpfg"
server_key: "6e85ea294acd85701523fe9f686afef7"
server_key_iv: "ad19a84d8e0ff789002e8138dd88ebf3"
server_key_version: "encryptionkey@0"
I20260812 03:08:34.256191 3268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:34.272316 3268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:34.275856 3268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:34.276088 3268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:34.277748 3268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:34.277817 3268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:34.277890 3268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:34.277921 3268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:08:34 dist-test-slave-zpfg krb5kdc[3145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504114, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM
I20260812 03:08:34.289383 3268 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:39723
I20260812 03:08:34.289558 3408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:39723 every 8 connection(s)
I20260812 03:08:34.290457 3268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:08:34.293586 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 3268
I20260812 03:08:34.293749 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance
I20260812 03:08:34.294071 27626 external_mini_cluster.cc:1428] Setting key 44afc00360e7af5a3f09d4b54240d4dd
I20260812 03:08:34.294417 3243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:34.278871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:37111 (local address 127.26.250.190:32913)
0812 03:08:34.279013 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:34.279017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:34.280229 (+ 1212us) server_negotiation.cc:407] Connection header received
0812 03:08:34.281531 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:34.281539 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:34.281610 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:34.281676 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:34.283413 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.284359 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.286454 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.286700 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.286788 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:34.290178 (+ 3390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:34.290182 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:34.290184 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:34.290212 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:34.292045 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:34.292606 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:34.292610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:34.292612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:34.292707 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:34.293086 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:34.293089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:34.293091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:34.293310 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:34.293426 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:34.294028 (+ 602us) server_negotiation.cc:299] Negotiation successful
0812 03:08:34.294194 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:08:34.295329 3294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:34.279521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:32913 (local address 127.26.250.129:37111)
0812 03:08:34.280066 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:34.280113 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:34.281208 (+ 1095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:34.282162 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:34.282169 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:34.282606 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:34.283244 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:34.283267 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.284536 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.284540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:34.286250 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:34.286264 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.286891 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.286895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:34.287138 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:34.287989 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:34.288015 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:34.289978 (+ 1963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:34.292184 (+ 2206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:34.292189 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:34.292206 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:34.292493 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:34.292814 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:34.292817 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:34.292820 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:34.292973 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:34.293424 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:34.293432 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:34.293713 (+ 281us) client_negotiation.cc:769] Sending connection context
0812 03:08:34.293971 (+ 258us) client_negotiation.cc:240] Negotiation successful
0812 03:08:34.294323 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":149,"threads_started":1}
I20260812 03:08:34.296810 3409 heartbeater.cc:347] Connected to a master server at 127.26.250.190:32913
I20260812 03:08:34.297062 3409 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:34.297750 3409 heartbeater.cc:511] Master 127.26.250.190:32913 requested a full tablet report, sending...
I20260812 03:08:34.299427 3192 ts_manager.cc:194] Registered new tserver with Master: 6801eb5071dc43bcbd45983831f97ce7 (127.26.250.129:39723)
I20260812 03:08:34.301098 3192 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.250.129@KRBTEST.COM'} at 127.26.250.129:37111
WARNING: no policy specified for oryx/127.26.250.130@KRBTEST.COM; defaulting to no policy
I20260812 03:08:34.308274 3243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:34.299714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:57703 (local address 127.26.250.190:32913)
0812 03:08:34.299831 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:34.299836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:34.300024 (+ 188us) server_negotiation.cc:407] Connection header received
0812 03:08:34.300170 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:34.300173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:34.300215 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:34.300313 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:34.300794 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.301460 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.302612 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.302961 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.303044 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:34.304541 (+ 1497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:34.304545 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:34.304548 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:34.304573 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:34.306295 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:34.306954 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:34.306957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:34.306959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:34.307040 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:34.307302 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:34.307329 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:34.307330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:34.307633 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:34.307727 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:34.308018 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 03:08:34.308148 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 03:08:34.308364 3417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:34.299623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:32913 (local address 127.26.250.129:57703)
0812 03:08:34.299928 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:34.299942 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:34.300072 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:34.300331 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:34.300333 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:34.300554 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:34.300669 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:34.300675 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.301600 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.301606 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:34.302464 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:34.302476 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.303214 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.303217 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:34.303310 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:34.303962 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:34.303976 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:34.304389 (+ 413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:34.306543 (+ 2154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:34.306549 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:34.306551 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:34.306843 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:34.307133 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:34.307135 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:34.307136 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:34.307219 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:34.307757 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:34.307760 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:34.307851 (+ 91us) client_negotiation.cc:769] Sending connection context
0812 03:08:34.307955 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 03:08:34.308082 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":107,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.250.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.250.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.130.keytab.
Entry for principal oryx/127.26.250.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.130.keytab.
I20260812 03:08:34.331071 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:0
--local_ip_for_outbound_sockets=127.26.250.130
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:32913
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.130.keytab
--principal=oryx/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:44285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:34.452843 3419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:34.453145 3419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:34.453195 3419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:34.457568 3419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:34.457660 3419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:34.457702 3419 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:08:34.457773 3419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:08:34.463738 3419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:44285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.130.keytab
--principal=oryx/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.250.190:32913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:34.464926 3419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:34.466044 3419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:34.476832 3426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:34.476946 3424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:34.476989 3419 server_base.cc:1034] running on GCE node
W20260812 03:08:34.476832 3428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:34.477725 3419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:34.478564 3419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:34.479769 3419 hybrid_clock.cc:648] HybridClock initialized: now 1786504114479727 us; error 54 us; skew 500 ppm
Aug 12 03:08:34 dist-test-slave-zpfg krb5kdc[3145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504114, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:34.482978 3419 init.cc:377] Logged in from keytab as oryx/127.26.250.130@KRBTEST.COM (short username oryx)
I20260812 03:08:34.484052 3419 webserver.cc:460] Webserver started at http://127.26.250.130:36629/ using document root <none> and password file <none>
I20260812 03:08:34.484695 3419 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:34.484802 3419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:34.485066 3419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:34.487484 3419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-1/data/instance:
uuid: "be9437917ecf496e852fe0f5bb10bec8"
format_stamp: "Formatted at 2026-08-12 03:08:34 on dist-test-slave-zpfg"
server_key: "da02c51da33ff239b586010d029da6f0"
server_key_iv: "89cf1fd644d655f5cbd30601c61c3e5e"
server_key_version: "encryptionkey@0"
I20260812 03:08:34.488205 3419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance:
uuid: "be9437917ecf496e852fe0f5bb10bec8"
format_stamp: "Formatted at 2026-08-12 03:08:34 on dist-test-slave-zpfg"
server_key: "da02c51da33ff239b586010d029da6f0"
server_key_iv: "89cf1fd644d655f5cbd30601c61c3e5e"
server_key_version: "encryptionkey@0"
I20260812 03:08:34.492508 3419 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:08:34.495795 3436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:34.497438 3419 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:08:34.497596 3419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "be9437917ecf496e852fe0f5bb10bec8"
format_stamp: "Formatted at 2026-08-12 03:08:34 on dist-test-slave-zpfg"
server_key: "da02c51da33ff239b586010d029da6f0"
server_key_iv: "89cf1fd644d655f5cbd30601c61c3e5e"
server_key_version: "encryptionkey@0"
I20260812 03:08:34.497802 3419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:34.520141 3419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:34.523921 3419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:34.524159 3419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:34.525837 3419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:34.525913 3419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:34.525982 3419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:34.526026 3419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:08:34 dist-test-slave-zpfg krb5kdc[3145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504114, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM
I20260812 03:08:34.537957 3419 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:45369
I20260812 03:08:34.538053 3557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:45369 every 8 connection(s)
I20260812 03:08:34.539124 3419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:08:34.539647 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 3419
I20260812 03:08:34.539772 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance
I20260812 03:08:34.540143 27626 external_mini_cluster.cc:1428] Setting key f028ef378915d8139fac2b2728b78cda
I20260812 03:08:34.542282 3243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:34.526952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:56295 (local address 127.26.250.190:32913)
0812 03:08:34.527089 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:34.527093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:34.528234 (+ 1141us) server_negotiation.cc:407] Connection header received
0812 03:08:34.529772 (+ 1538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:34.529777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:34.529850 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:34.529959 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:34.531609 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.532222 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.533758 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.534133 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.534278 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:34.537896 (+ 3618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:34.537900 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:34.537902 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:34.537939 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:34.539800 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:34.540455 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:34.540458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:34.540459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:34.540568 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:34.540947 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:34.540950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:34.540952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:34.541176 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:34.541277 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:34.541936 (+ 659us) server_negotiation.cc:299] Negotiation successful
0812 03:08:34.542070 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:08:34.543188 3445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:34.527550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:32913 (local address 127.26.250.130:56295)
0812 03:08:34.528056 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:34.528103 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:34.529445 (+ 1342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:34.530183 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:34.530195 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:34.530740 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:34.531395 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:34.531410 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.532417 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.532421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:34.533586 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:34.533596 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.534332 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.534359 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:34.534541 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:34.535799 (+ 1258us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:34.535815 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:34.537691 (+ 1876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:34.539956 (+ 2265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:34.539964 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:34.539981 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:34.540343 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:34.540681 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:34.540687 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:34.540692 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:34.540862 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:34.541350 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:34.541356 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:34.541690 (+ 334us) client_negotiation.cc:769] Sending connection context
0812 03:08:34.541857 (+ 167us) client_negotiation.cc:240] Negotiation successful
0812 03:08:34.542181 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":135,"threads_started":1}
I20260812 03:08:34.544600 3558 heartbeater.cc:347] Connected to a master server at 127.26.250.190:32913
I20260812 03:08:34.544780 3558 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:34.545397 3558 heartbeater.cc:511] Master 127.26.250.190:32913 requested a full tablet report, sending...
I20260812 03:08:34.546619 3191 ts_manager.cc:194] Registered new tserver with Master: be9437917ecf496e852fe0f5bb10bec8 (127.26.250.130:45369)
I20260812 03:08:34.547365 3191 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.250.130@KRBTEST.COM'} at 127.26.250.130:56295
WARNING: no policy specified for oryx/127.26.250.131@KRBTEST.COM; defaulting to no policy
I20260812 03:08:34.555176 3565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:34.546917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:32913 (local address 127.26.250.130:55643)
0812 03:08:34.547260 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:34.547276 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:34.547367 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:34.547694 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:34.547696 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:34.547931 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:34.548093 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:34.548104 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.548839 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.548842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:34.549568 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:34.549577 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.550041 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.550044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:34.550136 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:34.550786 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:34.550798 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:34.551277 (+ 479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:34.553350 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:34.553353 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:34.553355 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:34.553562 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:34.553862 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:34.553866 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:34.553868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:34.553956 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:34.554577 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:34.554580 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:34.554682 (+ 102us) client_negotiation.cc:769] Sending connection context
0812 03:08:34.554810 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 03:08:34.554943 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":3,"thread_start_us":123,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.250.131@KRBTEST.COM" created.
I20260812 03:08:34.555254 3243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:34.547302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:55643 (local address 127.26.250.190:32913)
0812 03:08:34.547438 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:34.547445 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:34.547463 (+ 18us) server_negotiation.cc:407] Connection header received
0812 03:08:34.547506 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:34.547509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:34.547559 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:34.547652 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:34.548201 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.548702 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.549696 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.549923 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.550036 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:34.551451 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:34.551455 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:34.551458 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:34.551492 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:34.553114 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:34.553667 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:34.553671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:34.553673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:34.553766 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:34.554049 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:34.554054 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:34.554055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:34.554282 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:34.554509 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:34.554906 (+ 397us) server_negotiation.cc:299] Negotiation successful
0812 03:08:34.555067 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.250.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.131.keytab.
Entry for principal oryx/127.26.250.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.131.keytab.
I20260812 03:08:34.574544 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:0
--local_ip_for_outbound_sockets=127.26.250.131
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:32913
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.131.keytab
--principal=oryx/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:44285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:34.697052 3567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:34.697470 3567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:34.697558 3567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:34.701938 3567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:34.702010 3567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:34.702081 3567 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 03:08:34.702148 3567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:08:34.707890 3567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:44285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.131.keytab
--principal=oryx/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.26.250.190:32913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:34.709098 3567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:34.710196 3567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:34.720816 3573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:34.720827 3575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:34.720945 3567 server_base.cc:1034] running on GCE node
W20260812 03:08:34.720839 3578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:34.721529 3567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:34.722257 3567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:34.723466 3567 hybrid_clock.cc:648] HybridClock initialized: now 1786504114723444 us; error 32 us; skew 500 ppm
Aug 12 03:08:34 dist-test-slave-zpfg krb5kdc[3145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504114, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:34.726657 3567 init.cc:377] Logged in from keytab as oryx/127.26.250.131@KRBTEST.COM (short username oryx)
I20260812 03:08:34.727609 3567 webserver.cc:460] Webserver started at http://127.26.250.131:38831/ using document root <none> and password file <none>
I20260812 03:08:34.728204 3567 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:34.728298 3567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:34.728539 3567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:34.730821 3567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-2/data/instance:
uuid: "32acdec1a00646eca00082ba4095be15"
format_stamp: "Formatted at 2026-08-12 03:08:34 on dist-test-slave-zpfg"
server_key: "197ebdd82912e7f1ca3a67ef13a9d2c7"
server_key_iv: "0c98bc6fe1d113d829ee79d3db777b12"
server_key_version: "encryptionkey@0"
I20260812 03:08:34.731467 3567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance:
uuid: "32acdec1a00646eca00082ba4095be15"
format_stamp: "Formatted at 2026-08-12 03:08:34 on dist-test-slave-zpfg"
server_key: "197ebdd82912e7f1ca3a67ef13a9d2c7"
server_key_iv: "0c98bc6fe1d113d829ee79d3db777b12"
server_key_version: "encryptionkey@0"
I20260812 03:08:34.735525 3567 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:08:34.738270 3585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:34.739607 3567 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:08:34.739749 3567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "32acdec1a00646eca00082ba4095be15"
format_stamp: "Formatted at 2026-08-12 03:08:34 on dist-test-slave-zpfg"
server_key: "197ebdd82912e7f1ca3a67ef13a9d2c7"
server_key_iv: "0c98bc6fe1d113d829ee79d3db777b12"
server_key_version: "encryptionkey@0"
I20260812 03:08:34.739873 3567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:34.749897 3567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:34.753383 3567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:34.753599 3567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:34.755203 3567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:34.755278 3567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:34.755358 3567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:34.755401 3567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 03:08:34 dist-test-slave-zpfg krb5kdc[3145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504114, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM
I20260812 03:08:34.766657 3567 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:40591
I20260812 03:08:34.766716 3705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:40591 every 8 connection(s)
I20260812 03:08:34.767964 3567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:08:34.770762 3243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:34.756123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:43793 (local address 127.26.250.190:32913)
0812 03:08:34.756322 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:34.756329 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:34.757196 (+ 867us) server_negotiation.cc:407] Connection header received
0812 03:08:34.758259 (+ 1063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:34.758266 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:34.758365 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:34.758475 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:34.760211 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.761160 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.763032 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.763381 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.763497 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:34.766487 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:34.766491 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:34.766493 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:34.766525 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:34.768458 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:34.769158 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:34.769164 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:34.769167 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:34.769270 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:34.769675 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:34.769677 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:34.769678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:34.769882 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:34.769971 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:34.770529 (+ 558us) server_negotiation.cc:299] Negotiation successful
0812 03:08:34.770638 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 03:08:34.771622 3594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:34.756537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:32913 (local address 127.26.250.131:43793)
0812 03:08:34.757016 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:34.757070 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:34.758004 (+ 934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:34.758707 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:34.758718 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:34.759283 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:34.760040 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:34.760064 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.761330 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.761334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:34.762880 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:34.762894 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.763556 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.763559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:34.763708 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:34.764434 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:34.764457 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:34.766223 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:34.768602 (+ 2379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:34.768609 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:34.768625 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:34.769038 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:34.769395 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:34.769398 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:34.769400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:34.769567 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:34.769984 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:34.769989 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:34.770226 (+ 237us) client_negotiation.cc:769] Sending connection context
0812 03:08:34.770490 (+ 264us) client_negotiation.cc:240] Negotiation successful
0812 03:08:34.770761 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":143,"threads_started":1}
I20260812 03:08:34.772964 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 3567
I20260812 03:08:34.773000 3706 heartbeater.cc:347] Connected to a master server at 127.26.250.190:32913
I20260812 03:08:34.773136 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance
I20260812 03:08:34.773155 3706 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:34.773546 27626 external_mini_cluster.cc:1428] Setting key 335497f20338cddbe0104dc53983f8ed
I20260812 03:08:34.778052 3706 heartbeater.cc:511] Master 127.26.250.190:32913 requested a full tablet report, sending...
I20260812 03:08:34.779609 3191 ts_manager.cc:194] Registered new tserver with Master: 32acdec1a00646eca00082ba4095be15 (127.26.250.131:40591)
I20260812 03:08:34.780455 3191 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.26.250.131@KRBTEST.COM'} at 127.26.250.131:43793
I20260812 03:08:34.788033 3243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:34.779051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:52989 (local address 127.26.250.190:32913)
0812 03:08:34.779509 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:34.779514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:34.780989 (+ 1475us) server_negotiation.cc:407] Connection header received
0812 03:08:34.781135 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:34.781138 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:34.781189 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:34.781273 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:34.781746 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.782262 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.783138 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.783371 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.783479 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:34.784691 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:34.784695 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:34.784698 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:34.784735 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:34.786163 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:34.786698 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:34.786701 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:34.786703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:34.786792 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:34.787088 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:34.787094 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:34.787096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:34.787291 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:34.787448 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:34.787729 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 03:08:34.787849 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374}
I20260812 03:08:34.788050 3712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:34.779033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:32913 (local address 127.26.250.131:52989)
0812 03:08:34.780894 (+ 1861us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:34.780916 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:34.781035 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:34.781292 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:34.781295 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:34.781518 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:34.781624 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:34.781630 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.782422 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.782426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:34.783001 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:34.783010 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.783488 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.783491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:34.783589 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:34.784102 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:34.784115 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:34.784514 (+ 399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:34.786296 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:34.786301 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:34.786303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:34.786580 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:34.786899 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:34.786903 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:34.786904 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:34.786996 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:34.787437 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:34.787440 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:34.787553 (+ 113us) client_negotiation.cc:769] Sending connection context
0812 03:08:34.787664 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 03:08:34.787839 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1749,"thread_start_us":111,"threads_started":1}
I20260812 03:08:34.791383 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 03:08:34 dist-test-slave-zpfg krb5kdc[3145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504114, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 03:08:34 dist-test-slave-zpfg krb5kdc[3145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:34 dist-test-slave-zpfg krb5kdc[3145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:08:34.817934 3243 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:34.811472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47316 (local address 127.26.250.190:32913)
0812 03:08:34.811607 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:34.811611 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:34.811623 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:08:34.811665 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:34.811667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:34.811706 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:34.811771 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:34.812666 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.813187 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.814031 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.814420 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.814526 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:34.817794 (+ 3268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47316: BlockingRecv error: recv got EOF from 127.0.0.1:47316 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 03:08:34 dist-test-slave-zpfg krb5kdc[3145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504114, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM
I20260812 03:08:34.827445 3243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:34.819516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47330 (local address 127.26.250.190:32913)
0812 03:08:34.819665 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:34.819668 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:34.819728 (+ 60us) server_negotiation.cc:407] Connection header received
0812 03:08:34.819823 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:34.819826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:34.819868 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:34.819937 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:34.820530 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.821121 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.821926 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.822173 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.822262 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:34.824570 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:34.824573 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:34.824575 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:34.824600 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:34.825855 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:34.826227 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:34.826229 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:34.826231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:34.826311 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:34.826604 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:34.826606 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:34.826607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:34.826785 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:34.826894 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:34.827095 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 03:08:34.827190 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 03:08:34.830322 3191 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:08:34.833451 3191 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:08:34.848207 3729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:34.842083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55738 (local address 127.26.250.130:45369)
0812 03:08:34.842579 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:34.842595 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:34.842624 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:08:34.842708 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:34.842714 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:34.842963 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:34.843132 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:34.843685 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.844938 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.846728 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.847511 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.847670 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:34.847830 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:34.847951 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 03:08:34.848044 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":139,"threads_started":1}
I20260812 03:08:34.848196 3728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:34.841942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:45369 (local address 127.0.0.1:55738)
0812 03:08:34.842300 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:34.842387 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:34.842520 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:34.843346 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:34.843353 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:34.843380 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:34.843540 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:34.843551 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.845109 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.845113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:34.846600 (+ 1487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:34.846610 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.847637 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.847641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:34.847751 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:34.847790 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 03:08:34.847912 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 03:08:34.848042 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":33,"thread_start_us":87,"threads_started":1}
I20260812 03:08:34.848474 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:34.843029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:39723 (local address 127.0.0.1:38076)
0812 03:08:34.843470 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:34.843485 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:34.843651 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:34.844432 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:34.844434 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:34.844446 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:34.844552 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:34.844560 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.845802 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.845818 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:34.846960 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:34.846967 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.848040 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.848045 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:34.848137 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:34.848160 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:08:34.848209 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 03:08:34.848286 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":45,"thread_start_us":103,"threads_started":1}
I20260812 03:08:34.849174 3733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:34.843509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38076 (local address 127.26.250.129:39723)
0812 03:08:34.843929 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:34.843938 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:34.843965 (+ 27us) server_negotiation.cc:407] Connection header received
0812 03:08:34.844042 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:34.844045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:34.844288 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:34.844547 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:34.844693 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.845676 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.847084 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.847890 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.848758 (+ 868us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:34.848883 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:34.848934 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 03:08:34.849037 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":101,"threads_started":1}
I20260812 03:08:34.849684 3731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:34.842974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52738 (local address 127.26.250.131:40591)
0812 03:08:34.843822 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:34.843845 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:34.843916 (+ 71us) server_negotiation.cc:407] Connection header received
0812 03:08:34.843995 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:34.844000 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:34.844186 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:34.844392 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:34.844796 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.846019 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.847258 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.847990 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.848117 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:34.848240 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:34.849453 (+ 1213us) server_negotiation.cc:299] Negotiation successful
0812 03:08:34.849547 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":738,"thread_start_us":122,"threads_started":1}
I20260812 03:08:34.849867 3730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:34.842706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:40591 (local address 127.0.0.1:52738)
0812 03:08:34.843402 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:34.843418 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:34.843504 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:34.844390 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:34.844395 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:34.844418 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:34.844690 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:34.844696 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.846116 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.846119 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:34.847129 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:34.847140 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.848843 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.848847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:34.848955 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:34.848977 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:08:34.849048 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 03:08:34.849110 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"mutex_wait_us":6,"thread_start_us":99,"threads_started":1}
I20260812 03:08:34.851410 3489 tablet_service.cc:1467] Processing CreateTablet for tablet 3584a1d72b844d96a209a89770ea07d6 (DEFAULT_TABLE table=test-table [id=f91c587da99441c0aa191172e67a2b17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:34.851791 3337 tablet_service.cc:1467] Processing CreateTablet for tablet 3584a1d72b844d96a209a89770ea07d6 (DEFAULT_TABLE table=test-table [id=f91c587da99441c0aa191172e67a2b17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:34.852638 3489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3584a1d72b844d96a209a89770ea07d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:34.852882 3337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3584a1d72b844d96a209a89770ea07d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:34.852556 3637 tablet_service.cc:1467] Processing CreateTablet for tablet 3584a1d72b844d96a209a89770ea07d6 (DEFAULT_TABLE table=test-table [id=f91c587da99441c0aa191172e67a2b17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:34.853574 3637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3584a1d72b844d96a209a89770ea07d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:34.860066 3735 tablet_bootstrap.cc:492] T 3584a1d72b844d96a209a89770ea07d6 P 6801eb5071dc43bcbd45983831f97ce7: Bootstrap starting.
I20260812 03:08:34.861030 3736 tablet_bootstrap.cc:492] T 3584a1d72b844d96a209a89770ea07d6 P be9437917ecf496e852fe0f5bb10bec8: Bootstrap starting.
I20260812 03:08:34.861972 3737 tablet_bootstrap.cc:492] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15: Bootstrap starting.
I20260812 03:08:34.862803 3735 tablet_bootstrap.cc:654] T 3584a1d72b844d96a209a89770ea07d6 P 6801eb5071dc43bcbd45983831f97ce7: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:34.863302 3736 tablet_bootstrap.cc:654] T 3584a1d72b844d96a209a89770ea07d6 P be9437917ecf496e852fe0f5bb10bec8: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:34.863742 3735 log.cc:824] T 3584a1d72b844d96a209a89770ea07d6 P 6801eb5071dc43bcbd45983831f97ce7: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:34.863902 3737 tablet_bootstrap.cc:654] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:34.864133 3736 log.cc:824] T 3584a1d72b844d96a209a89770ea07d6 P be9437917ecf496e852fe0f5bb10bec8: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:34.864766 3737 log.cc:824] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:34.866055 3735 tablet_bootstrap.cc:492] T 3584a1d72b844d96a209a89770ea07d6 P 6801eb5071dc43bcbd45983831f97ce7: No bootstrap required, opened a new log
I20260812 03:08:34.866274 3735 ts_tablet_manager.cc:1397] T 3584a1d72b844d96a209a89770ea07d6 P 6801eb5071dc43bcbd45983831f97ce7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:08:34.866506 3736 tablet_bootstrap.cc:492] T 3584a1d72b844d96a209a89770ea07d6 P be9437917ecf496e852fe0f5bb10bec8: No bootstrap required, opened a new log
I20260812 03:08:34.866710 3736 ts_tablet_manager.cc:1397] T 3584a1d72b844d96a209a89770ea07d6 P be9437917ecf496e852fe0f5bb10bec8: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 03:08:34.866732 3737 tablet_bootstrap.cc:492] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15: No bootstrap required, opened a new log
I20260812 03:08:34.866910 3737 ts_tablet_manager.cc:1397] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260812 03:08:34.869817 3736 raft_consensus.cc:348] T 3584a1d72b844d96a209a89770ea07d6 P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } } peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } }
I20260812 03:08:34.870085 3736 raft_consensus.cc:374] T 3584a1d72b844d96a209a89770ea07d6 P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:34.870138 3736 raft_consensus.cc:729] T 3584a1d72b844d96a209a89770ea07d6 P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9437917ecf496e852fe0f5bb10bec8, State: Initialized, Role: FOLLOWER
I20260812 03:08:34.870616 3736 consensus_queue.cc:260] T 3584a1d72b844d96a209a89770ea07d6 P be9437917ecf496e852fe0f5bb10bec8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } } peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } }
I20260812 03:08:34.871068 3735 raft_consensus.cc:348] T 3584a1d72b844d96a209a89770ea07d6 P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } } peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } }
I20260812 03:08:34.871068 3737 raft_consensus.cc:348] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } } peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } }
I20260812 03:08:34.871327 3737 raft_consensus.cc:374] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:34.871333 3735 raft_consensus.cc:374] T 3584a1d72b844d96a209a89770ea07d6 P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:34.871325 3558 heartbeater.cc:503] Master 127.26.250.190:32913 was elected leader, sending a full tablet report...
I20260812 03:08:34.871392 3735 raft_consensus.cc:729] T 3584a1d72b844d96a209a89770ea07d6 P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6801eb5071dc43bcbd45983831f97ce7, State: Initialized, Role: FOLLOWER
I20260812 03:08:34.871403 3737 raft_consensus.cc:729] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32acdec1a00646eca00082ba4095be15, State: Initialized, Role: FOLLOWER
I20260812 03:08:34.871831 3735 consensus_queue.cc:260] T 3584a1d72b844d96a209a89770ea07d6 P 6801eb5071dc43bcbd45983831f97ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } } peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } }
I20260812 03:08:34.871834 3737 consensus_queue.cc:260] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } } peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } }
I20260812 03:08:34.872506 3409 heartbeater.cc:503] Master 127.26.250.190:32913 was elected leader, sending a full tablet report...
I20260812 03:08:34.873133 3737 ts_tablet_manager.cc:1428] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 03:08:34.873436 3736 ts_tablet_manager.cc:1428] T 3584a1d72b844d96a209a89770ea07d6 P be9437917ecf496e852fe0f5bb10bec8: Time spent starting tablet: real 0.007s user 0.002s sys 0.003s
I20260812 03:08:34.873462 3706 heartbeater.cc:503] Master 127.26.250.190:32913 was elected leader, sending a full tablet report...
I20260812 03:08:34.874810 3735 ts_tablet_manager.cc:1428] T 3584a1d72b844d96a209a89770ea07d6 P 6801eb5071dc43bcbd45983831f97ce7: Time spent starting tablet: real 0.008s user 0.004s sys 0.002s
I20260812 03:08:34.927338 3744 raft_consensus.cc:482] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:08:34.927592 3744 raft_consensus.cc:504] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } } peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } }
I20260812 03:08:34.928907 3744 leader_election.cc:288] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9437917ecf496e852fe0f5bb10bec8 (127.26.250.130:45369), 6801eb5071dc43bcbd45983831f97ce7 (127.26.250.129:39723)
I20260812 03:08:34.933529 3594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:34.929176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:45369 (local address 127.26.250.131:34765)
0812 03:08:34.929320 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:34.929337 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:34.929446 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:34.929819 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:34.929822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:34.929843 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:34.929983 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:34.929989 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.931108 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.931110 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:34.932432 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:34.932444 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.933184 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.933187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:34.933277 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:34.933297 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 03:08:34.933363 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 03:08:34.933419 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 03:08:34.933627 3729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:34.929358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:34765 (local address 127.26.250.130:45369)
0812 03:08:34.929502 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:34.929506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:34.929520 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:08:34.929574 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:34.929576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:34.929615 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:34.929731 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:34.930105 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.930996 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.932576 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.933066 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.933193 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:34.933246 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:34.933410 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 03:08:34.933490 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:08:34.934211 3745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:34.929504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:39723 (local address 127.26.250.131:56837)
0812 03:08:34.929861 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:34.929874 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:34.929975 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:34.930277 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:34.930279 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:34.930294 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:34.930446 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:34.930451 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.931330 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.931334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:34.932796 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:34.932803 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.933831 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.933834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:34.933931 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:34.933955 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:08:34.934046 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 03:08:34.934100 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":138,"threads_started":1}
I20260812 03:08:34.934281 3733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:34.929777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:56837 (local address 127.26.250.129:39723)
0812 03:08:34.929920 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:34.929924 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:34.929940 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:08:34.930114 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:34.930117 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:34.930170 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:34.930257 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:34.930556 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.931228 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.932934 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.933550 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.933678 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:34.933732 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:34.934084 (+ 352us) server_negotiation.cc:299] Negotiation successful
0812 03:08:34.934169 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 03:08:34.934369 3512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3584a1d72b844d96a209a89770ea07d6" candidate_uuid: "32acdec1a00646eca00082ba4095be15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9437917ecf496e852fe0f5bb10bec8" is_pre_election: true
I20260812 03:08:34.934720 3512 raft_consensus.cc:2454] T 3584a1d72b844d96a209a89770ea07d6 P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32acdec1a00646eca00082ba4095be15 in term 0.
I20260812 03:08:34.935014 3359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3584a1d72b844d96a209a89770ea07d6" candidate_uuid: "32acdec1a00646eca00082ba4095be15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6801eb5071dc43bcbd45983831f97ce7" is_pre_election: true
I20260812 03:08:34.935330 3359 raft_consensus.cc:2454] T 3584a1d72b844d96a209a89770ea07d6 P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32acdec1a00646eca00082ba4095be15 in term 0.
I20260812 03:08:34.935315 3586 leader_election.cc:302] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32acdec1a00646eca00082ba4095be15, be9437917ecf496e852fe0f5bb10bec8; no voters:
I20260812 03:08:34.935608 3744 raft_consensus.cc:2781] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:08:34.935707 3744 raft_consensus.cc:482] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:08:34.935757 3744 raft_consensus.cc:3037] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:34.936887 3744 raft_consensus.cc:504] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } } peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } }
I20260812 03:08:34.937245 3744 leader_election.cc:288] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15 [CANDIDATE]: Term 1 election: Requested vote from peers be9437917ecf496e852fe0f5bb10bec8 (127.26.250.130:45369), 6801eb5071dc43bcbd45983831f97ce7 (127.26.250.129:39723)
I20260812 03:08:34.937685 3359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3584a1d72b844d96a209a89770ea07d6" candidate_uuid: "32acdec1a00646eca00082ba4095be15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6801eb5071dc43bcbd45983831f97ce7"
I20260812 03:08:34.937839 3359 raft_consensus.cc:3037] T 3584a1d72b844d96a209a89770ea07d6 P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:34.937636 3512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3584a1d72b844d96a209a89770ea07d6" candidate_uuid: "32acdec1a00646eca00082ba4095be15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9437917ecf496e852fe0f5bb10bec8"
I20260812 03:08:34.938019 3512 raft_consensus.cc:3037] T 3584a1d72b844d96a209a89770ea07d6 P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:34.939162 3359 raft_consensus.cc:2454] T 3584a1d72b844d96a209a89770ea07d6 P 6801eb5071dc43bcbd45983831f97ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32acdec1a00646eca00082ba4095be15 in term 1.
I20260812 03:08:34.939380 3512 raft_consensus.cc:2454] T 3584a1d72b844d96a209a89770ea07d6 P be9437917ecf496e852fe0f5bb10bec8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32acdec1a00646eca00082ba4095be15 in term 1.
I20260812 03:08:34.939587 3586 leader_election.cc:302] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32acdec1a00646eca00082ba4095be15, 6801eb5071dc43bcbd45983831f97ce7; no voters:
I20260812 03:08:34.939854 3744 raft_consensus.cc:2781] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:34.940128 3744 raft_consensus.cc:686] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15 [term 1 LEADER]: Becoming Leader. State: Replica: 32acdec1a00646eca00082ba4095be15, State: Running, Role: LEADER
I20260812 03:08:34.940526 3744 consensus_queue.cc:237] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } } peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } }
I20260812 03:08:34.944209 3191 catalog_manager.cc:5495] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32acdec1a00646eca00082ba4095be15 (127.26.250.131). New cstate: current_term: 1 leader_uuid: "32acdec1a00646eca00082ba4095be15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } health_report { overall_health: UNKNOWN } } }
I20260812 03:08:34.975605 3731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:34.971800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52754 (local address 127.26.250.131:40591)
0812 03:08:34.971961 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:34.971967 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:34.971986 (+ 19us) server_negotiation.cc:407] Connection header received
0812 03:08:34.972080 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:34.972083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:34.972136 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:34.972252 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:08:34.972515 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.973102 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.974034 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:34.974272 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:34.974413 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:34.974754 (+ 341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:08:34.975179 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:08:34.975317 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:34.975396 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 03:08:34.975458 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 03:08:34.981855 3359 raft_consensus.cc:1261] T 3584a1d72b844d96a209a89770ea07d6 P 6801eb5071dc43bcbd45983831f97ce7 [term 1 FOLLOWER]: Refusing update from remote peer 32acdec1a00646eca00082ba4095be15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:34.982368 3512 raft_consensus.cc:1261] T 3584a1d72b844d96a209a89770ea07d6 P be9437917ecf496e852fe0f5bb10bec8 [term 1 FOLLOWER]: Refusing update from remote peer 32acdec1a00646eca00082ba4095be15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:34.982892 3746 consensus_queue.cc:1035] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:34.983109 3744 consensus_queue.cc:1035] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:35.001724 3191 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:35.001986 3191 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:35.004895 3191 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf: Sending DeleteTablet for 3 replicas of tablet 3584a1d72b844d96a209a89770ea07d6
I20260812 03:08:35.006031 3337 tablet_service.cc:1514] Processing DeleteTablet for tablet 3584a1d72b844d96a209a89770ea07d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:34 UTC) from {username='oryx', principal='oryx/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:38076
I20260812 03:08:35.006107 3637 tablet_service.cc:1514] Processing DeleteTablet for tablet 3584a1d72b844d96a209a89770ea07d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:34 UTC) from {username='oryx', principal='oryx/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:52738
I20260812 03:08:35.006664 3759 tablet_replica.cc:330] T 3584a1d72b844d96a209a89770ea07d6 P 6801eb5071dc43bcbd45983831f97ce7: stopping tablet replica
I20260812 03:08:35.006860 3191 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 03:08:35.006937 3489 tablet_service.cc:1514] Processing DeleteTablet for tablet 3584a1d72b844d96a209a89770ea07d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:34 UTC) from {username='oryx', principal='oryx/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:55738
I20260812 03:08:35.007063 3759 raft_consensus.cc:2229] T 3584a1d72b844d96a209a89770ea07d6 P 6801eb5071dc43bcbd45983831f97ce7 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260812 03:08:35.007299 3191 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:08:35.007411 3759 raft_consensus.cc:2258] T 3584a1d72b844d96a209a89770ea07d6 P 6801eb5071dc43bcbd45983831f97ce7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:35.008522 3760 tablet_replica.cc:330] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15: stopping tablet replica
I20260812 03:08:35.008919 3760 raft_consensus.cc:2229] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:08:35.009189 3759 ts_tablet_manager.cc:1905] T 3584a1d72b844d96a209a89770ea07d6 P 6801eb5071dc43bcbd45983831f97ce7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:35.009454 3760 raft_consensus.cc:2258] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:35.010039 3761 tablet_replica.cc:330] T 3584a1d72b844d96a209a89770ea07d6 P be9437917ecf496e852fe0f5bb10bec8: stopping tablet replica
I20260812 03:08:35.010457 3761 raft_consensus.cc:2229] T 3584a1d72b844d96a209a89770ea07d6 P be9437917ecf496e852fe0f5bb10bec8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:35.010890 3761 raft_consensus.cc:2258] T 3584a1d72b844d96a209a89770ea07d6 P be9437917ecf496e852fe0f5bb10bec8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:35.012327 3759 ts_tablet_manager.cc:1918] T 3584a1d72b844d96a209a89770ea07d6 P 6801eb5071dc43bcbd45983831f97ce7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:08:35.012444 3759 log.cc:1196] T 3584a1d72b844d96a209a89770ea07d6 P 6801eb5071dc43bcbd45983831f97ce7: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-0/wal/wals/3584a1d72b844d96a209a89770ea07d6
I20260812 03:08:35.012817 3759 ts_tablet_manager.cc:1939] T 3584a1d72b844d96a209a89770ea07d6 P 6801eb5071dc43bcbd45983831f97ce7: Deleting consensus metadata
I20260812 03:08:35.014084 3337 tablet_service.cc:1467] Processing CreateTablet for tablet b2bc9547219d46ce8dcfbb2558e3f32a (DEFAULT_TABLE table=test-table [id=065d3f06174f4d82b09008770aa42c30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:35.014418 3337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2bc9547219d46ce8dcfbb2558e3f32a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:35.014565 3761 ts_tablet_manager.cc:1905] T 3584a1d72b844d96a209a89770ea07d6 P be9437917ecf496e852fe0f5bb10bec8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:35.016927 3735 tablet_bootstrap.cc:492] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7: Bootstrap starting.
I20260812 03:08:35.017230 3761 ts_tablet_manager.cc:1918] T 3584a1d72b844d96a209a89770ea07d6 P be9437917ecf496e852fe0f5bb10bec8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:08:35.017412 3761 log.cc:1196] T 3584a1d72b844d96a209a89770ea07d6 P be9437917ecf496e852fe0f5bb10bec8: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-1/wal/wals/3584a1d72b844d96a209a89770ea07d6
I20260812 03:08:35.017762 3761 ts_tablet_manager.cc:1939] T 3584a1d72b844d96a209a89770ea07d6 P be9437917ecf496e852fe0f5bb10bec8: Deleting consensus metadata
I20260812 03:08:35.017724 3489 tablet_service.cc:1467] Processing CreateTablet for tablet b2bc9547219d46ce8dcfbb2558e3f32a (DEFAULT_TABLE table=test-table [id=065d3f06174f4d82b09008770aa42c30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:35.018011 3489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2bc9547219d46ce8dcfbb2558e3f32a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:35.018057 3735 tablet_bootstrap.cc:654] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:35.018100 3637 tablet_service.cc:1467] Processing CreateTablet for tablet b2bc9547219d46ce8dcfbb2558e3f32a (DEFAULT_TABLE table=test-table [id=065d3f06174f4d82b09008770aa42c30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:35.018499 3637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2bc9547219d46ce8dcfbb2558e3f32a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:35.019577 3735 tablet_bootstrap.cc:492] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7: No bootstrap required, opened a new log
I20260812 03:08:35.019661 3735 ts_tablet_manager.cc:1397] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 03:08:35.020174 3735 raft_consensus.cc:348] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.020298 3735 raft_consensus.cc:374] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:35.020346 3735 raft_consensus.cc:729] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6801eb5071dc43bcbd45983831f97ce7, State: Initialized, Role: FOLLOWER
I20260812 03:08:35.020408 3736 tablet_bootstrap.cc:492] T b2bc9547219d46ce8dcfbb2558e3f32a P be9437917ecf496e852fe0f5bb10bec8: Bootstrap starting.
I20260812 03:08:35.021314 3175 catalog_manager.cc:4841] TS 6801eb5071dc43bcbd45983831f97ce7 (127.26.250.129:39723): tablet 3584a1d72b844d96a209a89770ea07d6 (table test-table [id=f91c587da99441c0aa191172e67a2b17]) successfully deleted
I20260812 03:08:35.021476 3736 tablet_bootstrap.cc:654] T b2bc9547219d46ce8dcfbb2558e3f32a P be9437917ecf496e852fe0f5bb10bec8: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:35.021983 3737 tablet_bootstrap.cc:492] T b2bc9547219d46ce8dcfbb2558e3f32a P 32acdec1a00646eca00082ba4095be15: Bootstrap starting.
I20260812 03:08:35.021926 3735 consensus_queue.cc:260] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.022428 3735 ts_tablet_manager.cc:1428] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260812 03:08:35.023134 3737 tablet_bootstrap.cc:654] T b2bc9547219d46ce8dcfbb2558e3f32a P 32acdec1a00646eca00082ba4095be15: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:35.024099 3736 tablet_bootstrap.cc:492] T b2bc9547219d46ce8dcfbb2558e3f32a P be9437917ecf496e852fe0f5bb10bec8: No bootstrap required, opened a new log
I20260812 03:08:35.024220 3736 ts_tablet_manager.cc:1397] T b2bc9547219d46ce8dcfbb2558e3f32a P be9437917ecf496e852fe0f5bb10bec8: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260812 03:08:35.024248 3175 catalog_manager.cc:4841] TS be9437917ecf496e852fe0f5bb10bec8 (127.26.250.130:45369): tablet 3584a1d72b844d96a209a89770ea07d6 (table test-table [id=f91c587da99441c0aa191172e67a2b17]) successfully deleted
I20260812 03:08:35.024703 3736 raft_consensus.cc:348] T b2bc9547219d46ce8dcfbb2558e3f32a P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.024834 3736 raft_consensus.cc:374] T b2bc9547219d46ce8dcfbb2558e3f32a P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:35.024876 3736 raft_consensus.cc:729] T b2bc9547219d46ce8dcfbb2558e3f32a P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9437917ecf496e852fe0f5bb10bec8, State: Initialized, Role: FOLLOWER
I20260812 03:08:35.025031 3736 consensus_queue.cc:260] T b2bc9547219d46ce8dcfbb2558e3f32a P be9437917ecf496e852fe0f5bb10bec8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.025321 3736 ts_tablet_manager.cc:1428] T b2bc9547219d46ce8dcfbb2558e3f32a P be9437917ecf496e852fe0f5bb10bec8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 03:08:35.028438 3760 ts_tablet_manager.cc:1905] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:35.031131 3737 tablet_bootstrap.cc:492] T b2bc9547219d46ce8dcfbb2558e3f32a P 32acdec1a00646eca00082ba4095be15: No bootstrap required, opened a new log
I20260812 03:08:35.031277 3737 ts_tablet_manager.cc:1397] T b2bc9547219d46ce8dcfbb2558e3f32a P 32acdec1a00646eca00082ba4095be15: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260812 03:08:35.031685 3760 ts_tablet_manager.cc:1918] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:08:35.031796 3760 log.cc:1196] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/3584a1d72b844d96a209a89770ea07d6
I20260812 03:08:35.031811 3737 raft_consensus.cc:348] T b2bc9547219d46ce8dcfbb2558e3f32a P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.031939 3737 raft_consensus.cc:374] T b2bc9547219d46ce8dcfbb2558e3f32a P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:35.031982 3737 raft_consensus.cc:729] T b2bc9547219d46ce8dcfbb2558e3f32a P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32acdec1a00646eca00082ba4095be15, State: Initialized, Role: FOLLOWER
I20260812 03:08:35.032133 3760 ts_tablet_manager.cc:1939] T 3584a1d72b844d96a209a89770ea07d6 P 32acdec1a00646eca00082ba4095be15: Deleting consensus metadata
I20260812 03:08:35.032132 3737 consensus_queue.cc:260] T b2bc9547219d46ce8dcfbb2558e3f32a P 32acdec1a00646eca00082ba4095be15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.032516 3737 ts_tablet_manager.cc:1428] T b2bc9547219d46ce8dcfbb2558e3f32a P 32acdec1a00646eca00082ba4095be15: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 03:08:35.033362 3175 catalog_manager.cc:4841] TS 32acdec1a00646eca00082ba4095be15 (127.26.250.131:40591): tablet 3584a1d72b844d96a209a89770ea07d6 (table test-table [id=f91c587da99441c0aa191172e67a2b17]) successfully deleted
W20260812 03:08:35.040596 3559 tablet.cc:2369] T b2bc9547219d46ce8dcfbb2558e3f32a P be9437917ecf496e852fe0f5bb10bec8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:08:35.042594 3411 tablet.cc:2369] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:08:35.270957 3707 tablet.cc:2369] T b2bc9547219d46ce8dcfbb2558e3f32a P 32acdec1a00646eca00082ba4095be15: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:08:35.342730 3743 raft_consensus.cc:482] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:08:35.342940 3743 raft_consensus.cc:504] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.344260 3743 leader_election.cc:288] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9437917ecf496e852fe0f5bb10bec8 (127.26.250.130:45369), 32acdec1a00646eca00082ba4095be15 (127.26.250.131:40591)
I20260812 03:08:35.348340 3729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:35.344714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:39455 (local address 127.26.250.130:45369)
0812 03:08:35.344936 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:35.344944 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:35.345039 (+ 95us) server_negotiation.cc:407] Connection header received
0812 03:08:35.345242 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:35.345246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:35.345323 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:35.345413 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:35.345685 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.346200 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.347324 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.347739 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.347905 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:35.347960 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:35.348137 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 03:08:35.348207 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 03:08:35.348361 3765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:35.344634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:45369 (local address 127.26.250.129:39455)
0812 03:08:35.344926 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:35.344949 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:35.345105 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:35.345436 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:35.345439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:35.345457 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:35.345562 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:35.345576 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.346327 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.346330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:35.347192 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:35.347202 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.347897 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.347901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:35.347992 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:35.348016 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:08:35.348116 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 03:08:35.348194 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":71,"threads_started":1}
I20260812 03:08:35.348764 3731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:35.345166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:53467 (local address 127.26.250.131:40591)
0812 03:08:35.345304 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:35.345309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:35.345509 (+ 200us) server_negotiation.cc:407] Connection header received
0812 03:08:35.345647 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:35.345650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:35.345705 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:35.345789 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:35.346086 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.346674 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.347712 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.348156 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.348287 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:35.348350 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:35.348559 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 03:08:35.348647 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 03:08:35.348861 3512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2bc9547219d46ce8dcfbb2558e3f32a" candidate_uuid: "6801eb5071dc43bcbd45983831f97ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9437917ecf496e852fe0f5bb10bec8" is_pre_election: true
I20260812 03:08:35.349045 3512 raft_consensus.cc:2454] T b2bc9547219d46ce8dcfbb2558e3f32a P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6801eb5071dc43bcbd45983831f97ce7 in term 0.
I20260812 03:08:35.349350 3768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:35.345038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:40591 (local address 127.26.250.129:53467)
0812 03:08:35.345418 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:35.345433 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:35.345544 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:35.345801 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:35.345804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:35.345819 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:35.345967 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:35.345973 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.346806 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.346809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:35.347581 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:35.347589 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.348280 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.348284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:35.348394 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:35.348423 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 03:08:35.349134 (+ 711us) client_negotiation.cc:240] Negotiation successful
0812 03:08:35.349234 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":132,"threads_started":1}
I20260812 03:08:35.349468 3286 leader_election.cc:302] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6801eb5071dc43bcbd45983831f97ce7, be9437917ecf496e852fe0f5bb10bec8; no voters:
I20260812 03:08:35.349792 3743 raft_consensus.cc:2781] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:08:35.349881 3743 raft_consensus.cc:482] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:08:35.349926 3743 raft_consensus.cc:3037] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:35.350193 3658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2bc9547219d46ce8dcfbb2558e3f32a" candidate_uuid: "6801eb5071dc43bcbd45983831f97ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32acdec1a00646eca00082ba4095be15" is_pre_election: true
I20260812 03:08:35.350482 3658 raft_consensus.cc:2454] T b2bc9547219d46ce8dcfbb2558e3f32a P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6801eb5071dc43bcbd45983831f97ce7 in term 0.
I20260812 03:08:35.351231 3743 raft_consensus.cc:504] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.351595 3743 leader_election.cc:288] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7 [CANDIDATE]: Term 1 election: Requested vote from peers be9437917ecf496e852fe0f5bb10bec8 (127.26.250.130:45369), 32acdec1a00646eca00082ba4095be15 (127.26.250.131:40591)
I20260812 03:08:35.351977 3512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2bc9547219d46ce8dcfbb2558e3f32a" candidate_uuid: "6801eb5071dc43bcbd45983831f97ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9437917ecf496e852fe0f5bb10bec8"
I20260812 03:08:35.352056 3658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2bc9547219d46ce8dcfbb2558e3f32a" candidate_uuid: "6801eb5071dc43bcbd45983831f97ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32acdec1a00646eca00082ba4095be15"
I20260812 03:08:35.352113 3512 raft_consensus.cc:3037] T b2bc9547219d46ce8dcfbb2558e3f32a P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:35.352202 3658 raft_consensus.cc:3037] T b2bc9547219d46ce8dcfbb2558e3f32a P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:35.353113 3512 raft_consensus.cc:2454] T b2bc9547219d46ce8dcfbb2558e3f32a P be9437917ecf496e852fe0f5bb10bec8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6801eb5071dc43bcbd45983831f97ce7 in term 1.
I20260812 03:08:35.353154 3658 raft_consensus.cc:2454] T b2bc9547219d46ce8dcfbb2558e3f32a P 32acdec1a00646eca00082ba4095be15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6801eb5071dc43bcbd45983831f97ce7 in term 1.
I20260812 03:08:35.353434 3286 leader_election.cc:302] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6801eb5071dc43bcbd45983831f97ce7, be9437917ecf496e852fe0f5bb10bec8; no voters:
I20260812 03:08:35.353654 3743 raft_consensus.cc:2781] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:35.353886 3743 raft_consensus.cc:686] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7 [term 1 LEADER]: Becoming Leader. State: Replica: 6801eb5071dc43bcbd45983831f97ce7, State: Running, Role: LEADER
I20260812 03:08:35.354151 3743 consensus_queue.cc:237] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.356932 3191 catalog_manager.cc:5495] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6801eb5071dc43bcbd45983831f97ce7 (127.26.250.129). New cstate: current_term: 1 leader_uuid: "6801eb5071dc43bcbd45983831f97ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } health_report { overall_health: UNKNOWN } } }
Aug 12 03:08:35 dist-test-slave-zpfg krb5kdc[3145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504113, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.190@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM
I20260812 03:08:35.409610 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:35.400375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47340 (local address 127.26.250.190:32913)
0812 03:08:35.400653 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:35.400660 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:35.400803 (+ 143us) server_negotiation.cc:407] Connection header received
0812 03:08:35.400927 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:35.400930 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:35.400971 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:35.401091 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:35.401843 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.402515 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.403306 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.403576 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.403689 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:35.406295 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:35.406299 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:35.406301 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:35.406374 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:35.407766 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:35.408249 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:35.408254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:35.408256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:35.408345 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:35.408622 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:35.408625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:35.408627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:35.408815 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:35.408919 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:35.409230 (+ 311us) server_negotiation.cc:299] Negotiation successful
0812 03:08:35.409426 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":112,"server-negotiator.queue_time_us":203,"thread_start_us":118,"threads_started":1}
I20260812 03:08:35.409610 3777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:35.400334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:32913 (local address 127.0.0.1:47340)
0812 03:08:35.400719 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:35.400733 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:35.400844 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:35.401109 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:35.401112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:35.401629 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:35.401728 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:35.401735 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.402630 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.402633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:35.403187 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:35.403195 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.403694 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.403697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:35.403810 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:35.404486 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:35.404499 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:35.406149 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:35.407889 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:35.407901 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:35.407908 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:35.408149 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:35.408448 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:35.408451 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:35.408453 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:35.408531 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:35.408930 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:35.408939 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:35.409060 (+ 121us) client_negotiation.cc:769] Sending connection context
0812 03:08:35.409176 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 03:08:35.409365 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":22,"thread_start_us":142,"threads_started":1}
I20260812 03:08:35.413760 3191 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:47340:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 03:08:35.414510 3191 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:08:35.419572 3637 tablet_service.cc:1467] Processing CreateTablet for tablet 03846005d7464dd6a0fc7f7760968e8b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2a40b4d285b642819c999afb5349b862]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:08:35.419528 3337 tablet_service.cc:1467] Processing CreateTablet for tablet 03846005d7464dd6a0fc7f7760968e8b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2a40b4d285b642819c999afb5349b862]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:08:35.419754 3489 tablet_service.cc:1467] Processing CreateTablet for tablet 03846005d7464dd6a0fc7f7760968e8b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2a40b4d285b642819c999afb5349b862]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 03:08:35.419975 3337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03846005d7464dd6a0fc7f7760968e8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:35.419975 3637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03846005d7464dd6a0fc7f7760968e8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:35.420233 3489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03846005d7464dd6a0fc7f7760968e8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:35.423027 3735 tablet_bootstrap.cc:492] T 03846005d7464dd6a0fc7f7760968e8b P 6801eb5071dc43bcbd45983831f97ce7: Bootstrap starting.
I20260812 03:08:35.423024 3737 tablet_bootstrap.cc:492] T 03846005d7464dd6a0fc7f7760968e8b P 32acdec1a00646eca00082ba4095be15: Bootstrap starting.
I20260812 03:08:35.423311 3736 tablet_bootstrap.cc:492] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8: Bootstrap starting.
I20260812 03:08:35.423954 3737 tablet_bootstrap.cc:654] T 03846005d7464dd6a0fc7f7760968e8b P 32acdec1a00646eca00082ba4095be15: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:35.424065 3735 tablet_bootstrap.cc:654] T 03846005d7464dd6a0fc7f7760968e8b P 6801eb5071dc43bcbd45983831f97ce7: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:35.424255 3736 tablet_bootstrap.cc:654] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:35.425752 3736 tablet_bootstrap.cc:492] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8: No bootstrap required, opened a new log
I20260812 03:08:35.425868 3737 tablet_bootstrap.cc:492] T 03846005d7464dd6a0fc7f7760968e8b P 32acdec1a00646eca00082ba4095be15: No bootstrap required, opened a new log
I20260812 03:08:35.425868 3736 ts_tablet_manager.cc:1397] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260812 03:08:35.425937 3737 ts_tablet_manager.cc:1397] T 03846005d7464dd6a0fc7f7760968e8b P 32acdec1a00646eca00082ba4095be15: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 03:08:35.426215 3735 tablet_bootstrap.cc:492] T 03846005d7464dd6a0fc7f7760968e8b P 6801eb5071dc43bcbd45983831f97ce7: No bootstrap required, opened a new log
I20260812 03:08:35.426291 3735 ts_tablet_manager.cc:1397] T 03846005d7464dd6a0fc7f7760968e8b P 6801eb5071dc43bcbd45983831f97ce7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:08:35.426267 3737 raft_consensus.cc:348] T 03846005d7464dd6a0fc7f7760968e8b P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.426448 3737 raft_consensus.cc:374] T 03846005d7464dd6a0fc7f7760968e8b P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:35.426393 3736 raft_consensus.cc:348] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.426497 3737 raft_consensus.cc:729] T 03846005d7464dd6a0fc7f7760968e8b P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32acdec1a00646eca00082ba4095be15, State: Initialized, Role: FOLLOWER
I20260812 03:08:35.426509 3736 raft_consensus.cc:374] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:35.426541 3736 raft_consensus.cc:729] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9437917ecf496e852fe0f5bb10bec8, State: Initialized, Role: FOLLOWER
I20260812 03:08:35.426626 3737 consensus_queue.cc:260] T 03846005d7464dd6a0fc7f7760968e8b P 32acdec1a00646eca00082ba4095be15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.426824 3735 raft_consensus.cc:348] T 03846005d7464dd6a0fc7f7760968e8b P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.426942 3735 raft_consensus.cc:374] T 03846005d7464dd6a0fc7f7760968e8b P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:35.426941 3737 ts_tablet_manager.cc:1428] T 03846005d7464dd6a0fc7f7760968e8b P 32acdec1a00646eca00082ba4095be15: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:08:35.426975 3735 raft_consensus.cc:729] T 03846005d7464dd6a0fc7f7760968e8b P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6801eb5071dc43bcbd45983831f97ce7, State: Initialized, Role: FOLLOWER
I20260812 03:08:35.426916 3756 tablet_replica.cc:439] T 03846005d7464dd6a0fc7f7760968e8b P 32acdec1a00646eca00082ba4095be15: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } } }
I20260812 03:08:35.427062 3756 tablet_replica.cc:442] T 03846005d7464dd6a0fc7f7760968e8b P 32acdec1a00646eca00082ba4095be15: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:08:35.427096 3735 consensus_queue.cc:260] T 03846005d7464dd6a0fc7f7760968e8b P 6801eb5071dc43bcbd45983831f97ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.426703 3736 consensus_queue.cc:260] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.427376 3735 ts_tablet_manager.cc:1428] T 03846005d7464dd6a0fc7f7760968e8b P 6801eb5071dc43bcbd45983831f97ce7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:08:35.427366 3743 tablet_replica.cc:439] T 03846005d7464dd6a0fc7f7760968e8b P 6801eb5071dc43bcbd45983831f97ce7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } } }
I20260812 03:08:35.427397 3741 tablet_replica.cc:439] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } } }
I20260812 03:08:35.427531 3741 tablet_replica.cc:442] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:08:35.427531 3743 tablet_replica.cc:442] T 03846005d7464dd6a0fc7f7760968e8b P 6801eb5071dc43bcbd45983831f97ce7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:08:35.429646 3736 ts_tablet_manager.cc:1428] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260812 03:08:35.570247 3741 raft_consensus.cc:482] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:08:35.570465 3741 raft_consensus.cc:504] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.571786 3741 leader_election.cc:288] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6801eb5071dc43bcbd45983831f97ce7 (127.26.250.129:39723), 32acdec1a00646eca00082ba4095be15 (127.26.250.131:40591)
I20260812 03:08:35.576166 3731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:35.572594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:40943 (local address 127.26.250.131:40591)
0812 03:08:35.572755 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:35.572762 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:35.572805 (+ 43us) server_negotiation.cc:407] Connection header received
0812 03:08:35.572993 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:35.572997 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:35.573055 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:35.573131 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:35.573413 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.573963 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.575137 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.575533 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.575669 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:35.575731 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:35.575938 (+ 207us) server_negotiation.cc:299] Negotiation successful
0812 03:08:35.576014 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:08:35.576193 3781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:35.571990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:39723 (local address 127.26.250.130:33135)
0812 03:08:35.572310 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:35.572324 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:35.572462 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:35.572788 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:35.572792 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:35.572816 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:35.572998 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:35.573007 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.573919 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.573923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:35.574924 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:35.574933 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.575706 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.575710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:35.575819 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:35.575844 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 03:08:35.575935 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 03:08:35.576010 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":2,"spinlock_wait_cycles":8960,"thread_start_us":90,"threads_started":1}
I20260812 03:08:35.576195 3783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:35.572410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:40591 (local address 127.26.250.130:40943)
0812 03:08:35.572745 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:35.572758 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:35.572852 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:35.573163 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:35.573166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:35.573185 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:08:35.573275 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:35.573280 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.574102 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.574106 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:35.574977 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:35.575009 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.575663 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.575667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:35.575772 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:35.575832 (+ 60us) client_negotiation.cc:769] Sending connection context
0812 03:08:35.575894 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 03:08:35.575992 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":55,"thread_start_us":53,"threads_started":1}
I20260812 03:08:35.576186 3782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:35.572134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:33135 (local address 127.26.250.129:39723)
0812 03:08:35.572438 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:35.572442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:35.572460 (+ 18us) server_negotiation.cc:407] Connection header received
0812 03:08:35.572603 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:35.572606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:35.572662 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:35.572758 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:08:35.573137 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.573801 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.575061 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.575539 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.575679 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:35.575738 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:35.575974 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 03:08:35.576054 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":94,"threads_started":1}
I20260812 03:08:35.576721 3658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03846005d7464dd6a0fc7f7760968e8b" candidate_uuid: "be9437917ecf496e852fe0f5bb10bec8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32acdec1a00646eca00082ba4095be15" is_pre_election: true
I20260812 03:08:35.576910 3658 raft_consensus.cc:2454] T 03846005d7464dd6a0fc7f7760968e8b P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9437917ecf496e852fe0f5bb10bec8 in term 0.
I20260812 03:08:35.577016 3359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03846005d7464dd6a0fc7f7760968e8b" candidate_uuid: "be9437917ecf496e852fe0f5bb10bec8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6801eb5071dc43bcbd45983831f97ce7" is_pre_election: true
I20260812 03:08:35.577196 3359 raft_consensus.cc:2454] T 03846005d7464dd6a0fc7f7760968e8b P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9437917ecf496e852fe0f5bb10bec8 in term 0.
I20260812 03:08:35.577312 3437 leader_election.cc:302] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32acdec1a00646eca00082ba4095be15, be9437917ecf496e852fe0f5bb10bec8; no voters:
I20260812 03:08:35.577596 3741 raft_consensus.cc:2781] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:08:35.577654 3741 raft_consensus.cc:482] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:08:35.577682 3741 raft_consensus.cc:3037] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:35.578804 3741 raft_consensus.cc:504] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.579173 3741 leader_election.cc:288] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8 [CANDIDATE]: Term 1 election: Requested vote from peers 6801eb5071dc43bcbd45983831f97ce7 (127.26.250.129:39723), 32acdec1a00646eca00082ba4095be15 (127.26.250.131:40591)
I20260812 03:08:35.579715 3359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03846005d7464dd6a0fc7f7760968e8b" candidate_uuid: "be9437917ecf496e852fe0f5bb10bec8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6801eb5071dc43bcbd45983831f97ce7"
I20260812 03:08:35.579715 3658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03846005d7464dd6a0fc7f7760968e8b" candidate_uuid: "be9437917ecf496e852fe0f5bb10bec8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32acdec1a00646eca00082ba4095be15"
I20260812 03:08:35.579864 3359 raft_consensus.cc:3037] T 03846005d7464dd6a0fc7f7760968e8b P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:35.579864 3658 raft_consensus.cc:3037] T 03846005d7464dd6a0fc7f7760968e8b P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:35.581233 3359 raft_consensus.cc:2454] T 03846005d7464dd6a0fc7f7760968e8b P 6801eb5071dc43bcbd45983831f97ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9437917ecf496e852fe0f5bb10bec8 in term 1.
I20260812 03:08:35.581346 3658 raft_consensus.cc:2454] T 03846005d7464dd6a0fc7f7760968e8b P 32acdec1a00646eca00082ba4095be15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9437917ecf496e852fe0f5bb10bec8 in term 1.
I20260812 03:08:35.581632 3437 leader_election.cc:302] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6801eb5071dc43bcbd45983831f97ce7, be9437917ecf496e852fe0f5bb10bec8; no voters:
I20260812 03:08:35.581925 3741 raft_consensus.cc:2781] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:35.582145 3741 raft_consensus.cc:686] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8 [term 1 LEADER]: Becoming Leader. State: Replica: be9437917ecf496e852fe0f5bb10bec8, State: Running, Role: LEADER
I20260812 03:08:35.582562 3741 consensus_queue.cc:237] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.583936 3784 tablet_replica.cc:439] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8: TxnStatusTablet state changed. Reason: New leader be9437917ecf496e852fe0f5bb10bec8. Latest consensus state: current_term: 1 leader_uuid: "be9437917ecf496e852fe0f5bb10bec8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } } }
I20260812 03:08:35.584067 3784 tablet_replica.cc:442] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:08:35.584394 3786 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 03:08:35.584563 3786 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 03:08:35.585556 3190 catalog_manager.cc:5495] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8 reported cstate change: term changed from 0 to 1, leader changed from <none> to be9437917ecf496e852fe0f5bb10bec8 (127.26.250.130). New cstate: current_term: 1 leader_uuid: "be9437917ecf496e852fe0f5bb10bec8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } health_report { overall_health: UNKNOWN } } }
I20260812 03:08:35.618459 3729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:35.614550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55754 (local address 127.26.250.130:45369)
0812 03:08:35.614695 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:35.614700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:35.614744 (+ 44us) server_negotiation.cc:407] Connection header received
0812 03:08:35.614973 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:35.614975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:35.615034 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:35.615140 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:08:35.615409 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.615918 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.616801 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.617028 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.617140 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:35.617455 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:08:35.618011 (+ 556us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:08:35.618156 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:35.618226 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 03:08:35.618283 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 03:08:35.618459 3777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:35.614425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:45369 (local address 127.0.0.1:55754)
0812 03:08:35.614679 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:35.614695 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:35.614818 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:35.615168 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:35.615171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:35.615179 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:08:35.615297 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:35.615308 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.616035 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.616039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:35.616681 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:35.616690 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.617144 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.617147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:35.617239 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:35.617290 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:08:35.618125 (+ 835us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:08:35.618142 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 03:08:35.618215 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 03:08:35.618318 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260812 03:08:35.621721 3359 raft_consensus.cc:1261] T 03846005d7464dd6a0fc7f7760968e8b P 6801eb5071dc43bcbd45983831f97ce7 [term 1 FOLLOWER]: Refusing update from remote peer be9437917ecf496e852fe0f5bb10bec8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:35.622097 3658 raft_consensus.cc:1261] T 03846005d7464dd6a0fc7f7760968e8b P 32acdec1a00646eca00082ba4095be15 [term 1 FOLLOWER]: Refusing update from remote peer be9437917ecf496e852fe0f5bb10bec8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:35.622411 3784 consensus_queue.cc:1035] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:35.622668 3741 consensus_queue.cc:1035] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:35.624249 3743 tablet_replica.cc:439] T 03846005d7464dd6a0fc7f7760968e8b P 6801eb5071dc43bcbd45983831f97ce7: TxnStatusTablet state changed. Reason: New leader be9437917ecf496e852fe0f5bb10bec8. Latest consensus state: current_term: 1 leader_uuid: "be9437917ecf496e852fe0f5bb10bec8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } } }
I20260812 03:08:35.624372 3743 tablet_replica.cc:442] T 03846005d7464dd6a0fc7f7760968e8b P 6801eb5071dc43bcbd45983831f97ce7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:08:35.624501 3756 tablet_replica.cc:439] T 03846005d7464dd6a0fc7f7760968e8b P 32acdec1a00646eca00082ba4095be15: TxnStatusTablet state changed. Reason: New leader be9437917ecf496e852fe0f5bb10bec8. Latest consensus state: current_term: 1 leader_uuid: "be9437917ecf496e852fe0f5bb10bec8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } } }
I20260812 03:08:35.624610 3756 tablet_replica.cc:442] T 03846005d7464dd6a0fc7f7760968e8b P 32acdec1a00646eca00082ba4095be15: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:08:35.626504 3784 tablet_replica.cc:439] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "be9437917ecf496e852fe0f5bb10bec8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } } }
I20260812 03:08:35.626645 3784 tablet_replica.cc:442] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:08:35.626838 3789 mvcc.cc:205] Tried to move back new op lower bound from 7317520857581047808 to 7317520857429172224. Current Snapshot: MvccSnapshot[applied={T|T < 7317520857581047808}]
I20260812 03:08:35.627861 3741 tablet_replica.cc:439] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "be9437917ecf496e852fe0f5bb10bec8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } } }
I20260812 03:08:35.627971 3741 tablet_replica.cc:442] T 03846005d7464dd6a0fc7f7760968e8b P be9437917ecf496e852fe0f5bb10bec8: This TxnStatusTablet replica's current role is: LEADER
I20260812 03:08:35.628121 3756 tablet_replica.cc:439] T 03846005d7464dd6a0fc7f7760968e8b P 32acdec1a00646eca00082ba4095be15: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "be9437917ecf496e852fe0f5bb10bec8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } } }
I20260812 03:08:35.628226 3756 tablet_replica.cc:442] T 03846005d7464dd6a0fc7f7760968e8b P 32acdec1a00646eca00082ba4095be15: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:08:35.629315 3791 mvcc.cc:205] Tried to move back new op lower bound from 7317520857581047808 to 7317520857429172224. Current Snapshot: MvccSnapshot[applied={T|T < 7317520857581047808}]
I20260812 03:08:35.630457 3743 tablet_replica.cc:439] T 03846005d7464dd6a0fc7f7760968e8b P 6801eb5071dc43bcbd45983831f97ce7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "be9437917ecf496e852fe0f5bb10bec8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } } }
I20260812 03:08:35.630577 3743 tablet_replica.cc:442] T 03846005d7464dd6a0fc7f7760968e8b P 6801eb5071dc43bcbd45983831f97ce7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 03:08:35.637355 3782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:35.633282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38084 (local address 127.26.250.129:39723)
0812 03:08:35.633421 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:35.633425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:35.633597 (+ 172us) server_negotiation.cc:407] Connection header received
0812 03:08:35.633780 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:35.633784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:35.633847 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:35.633955 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:08:35.634210 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.634941 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.635832 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.636076 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.636207 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:35.636496 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:08:35.636952 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:08:35.637085 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:35.637148 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 03:08:35.637207 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 03:08:35.648936 3729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:35.645051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:53479 (local address 127.26.250.130:45369)
0812 03:08:35.645230 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:35.645234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:35.645370 (+ 136us) server_negotiation.cc:407] Connection header received
0812 03:08:35.645498 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:35.645500 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:35.645545 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:35.645608 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:08:35.645910 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.646501 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.647826 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.648042 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.648124 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:35.648449 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:08:35.648520 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:08:35.648573 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:35.648755 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 03:08:35.648818 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260812 03:08:35.648975 3802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:35.644916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:45369 (local address 127.26.250.129:53479)
0812 03:08:35.645268 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:35.645283 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:35.645393 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:35.645638 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:35.645641 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:35.645649 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:08:35.645783 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:35.645789 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.646601 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.646604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:35.647695 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:35.647707 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.648154 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.648158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:35.648259 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:35.648314 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:08:35.648600 (+ 286us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:08:35.648606 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:08:35.648675 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 03:08:35.648767 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":102,"threads_started":1}
I20260812 03:08:35.656886 3658 raft_consensus.cc:1261] T b2bc9547219d46ce8dcfbb2558e3f32a P 32acdec1a00646eca00082ba4095be15 [term 1 FOLLOWER]: Refusing update from remote peer 6801eb5071dc43bcbd45983831f97ce7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:35.656992 3512 raft_consensus.cc:1261] T b2bc9547219d46ce8dcfbb2558e3f32a P be9437917ecf496e852fe0f5bb10bec8 [term 1 FOLLOWER]: Refusing update from remote peer 6801eb5071dc43bcbd45983831f97ce7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:35.657716 3769 consensus_queue.cc:1035] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:35.657975 3743 consensus_queue.cc:1035] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:35.661715 3790 mvcc.cc:205] Tried to move back new op lower bound from 7317520857726062592 to 7317520856493305856. Current Snapshot: MvccSnapshot[applied={T|T < 7317520857726062592}]
I20260812 03:08:35.682632 3782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:35.678679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:49569 (local address 127.26.250.129:39723)
0812 03:08:35.678859 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:35.678865 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:35.679173 (+ 308us) server_negotiation.cc:407] Connection header received
0812 03:08:35.679307 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:35.679310 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:35.679366 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:35.679446 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:08:35.679677 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.680457 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.681410 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.681689 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.681794 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:35.682138 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:08:35.682220 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:08:35.682261 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:35.682461 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 03:08:35.682524 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:08:35.682749 3808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:35.678592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:39723 (local address 127.26.250.130:49569)
0812 03:08:35.679067 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:35.679084 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:35.679199 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:35.679474 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:35.679477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:35.679486 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 03:08:35.679594 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:35.679599 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.680564 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.680566 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:35.681273 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:35.681284 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.681809 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.681813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:35.681927 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:35.681981 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 03:08:35.682318 (+ 337us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 03:08:35.682323 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 03:08:35.682496 (+ 173us) client_negotiation.cc:240] Negotiation successful
0812 03:08:35.682582 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":107,"threads_started":1}
I20260812 03:08:35.707794 3189 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:35.707989 3189 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:35.710201 3189 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf: Sending DeleteTablet for 3 replicas of tablet b2bc9547219d46ce8dcfbb2558e3f32a
I20260812 03:08:35.710983 3637 tablet_service.cc:1514] Processing DeleteTablet for tablet b2bc9547219d46ce8dcfbb2558e3f32a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:35 UTC) from {username='oryx', principal='oryx/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:52738
I20260812 03:08:35.711439 3811 tablet_replica.cc:330] T b2bc9547219d46ce8dcfbb2558e3f32a P 32acdec1a00646eca00082ba4095be15: stopping tablet replica
I20260812 03:08:35.711622 3337 tablet_service.cc:1514] Processing DeleteTablet for tablet b2bc9547219d46ce8dcfbb2558e3f32a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:35 UTC) from {username='oryx', principal='oryx/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:38076
I20260812 03:08:35.711678 3811 raft_consensus.cc:2229] T b2bc9547219d46ce8dcfbb2558e3f32a P 32acdec1a00646eca00082ba4095be15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:35.711963 3811 raft_consensus.cc:2258] T b2bc9547219d46ce8dcfbb2558e3f32a P 32acdec1a00646eca00082ba4095be15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:35.712383 3489 tablet_service.cc:1514] Processing DeleteTablet for tablet b2bc9547219d46ce8dcfbb2558e3f32a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:35 UTC) from {username='oryx', principal='oryx/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:55738
I20260812 03:08:35.712793 3812 tablet_replica.cc:330] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7: stopping tablet replica
I20260812 03:08:35.712955 3812 raft_consensus.cc:2229] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:08:35.713282 3812 raft_consensus.cc:2258] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:35.714108 3812 ts_tablet_manager.cc:1905] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:35.717589 3811 ts_tablet_manager.cc:1905] T b2bc9547219d46ce8dcfbb2558e3f32a P 32acdec1a00646eca00082ba4095be15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:35.718405 3812 ts_tablet_manager.cc:1918] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:08:35.718509 3812 log.cc:1196] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-0/wal/wals/b2bc9547219d46ce8dcfbb2558e3f32a
I20260812 03:08:35.718850 3812 ts_tablet_manager.cc:1939] T b2bc9547219d46ce8dcfbb2558e3f32a P 6801eb5071dc43bcbd45983831f97ce7: Deleting consensus metadata
I20260812 03:08:35.719794 3175 catalog_manager.cc:4841] TS 6801eb5071dc43bcbd45983831f97ce7 (127.26.250.129:39723): tablet b2bc9547219d46ce8dcfbb2558e3f32a (table test-table [id=065d3f06174f4d82b09008770aa42c30]) successfully deleted
I20260812 03:08:35.720242 3813 tablet_replica.cc:330] T b2bc9547219d46ce8dcfbb2558e3f32a P be9437917ecf496e852fe0f5bb10bec8: stopping tablet replica
I20260812 03:08:35.720412 3813 raft_consensus.cc:2229] T b2bc9547219d46ce8dcfbb2558e3f32a P be9437917ecf496e852fe0f5bb10bec8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:35.720582 3813 raft_consensus.cc:2258] T b2bc9547219d46ce8dcfbb2558e3f32a P be9437917ecf496e852fe0f5bb10bec8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:35.721460 3813 ts_tablet_manager.cc:1905] T b2bc9547219d46ce8dcfbb2558e3f32a P be9437917ecf496e852fe0f5bb10bec8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:35.722033 3811 ts_tablet_manager.cc:1918] T b2bc9547219d46ce8dcfbb2558e3f32a P 32acdec1a00646eca00082ba4095be15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:08:35.722193 3811 log.cc:1196] T b2bc9547219d46ce8dcfbb2558e3f32a P 32acdec1a00646eca00082ba4095be15: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/b2bc9547219d46ce8dcfbb2558e3f32a
I20260812 03:08:35.722572 3811 ts_tablet_manager.cc:1939] T b2bc9547219d46ce8dcfbb2558e3f32a P 32acdec1a00646eca00082ba4095be15: Deleting consensus metadata
I20260812 03:08:35.723527 3175 catalog_manager.cc:4841] TS 32acdec1a00646eca00082ba4095be15 (127.26.250.131:40591): tablet b2bc9547219d46ce8dcfbb2558e3f32a (table test-table [id=065d3f06174f4d82b09008770aa42c30]) successfully deleted
I20260812 03:08:35.724721 3813 ts_tablet_manager.cc:1918] T b2bc9547219d46ce8dcfbb2558e3f32a P be9437917ecf496e852fe0f5bb10bec8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:08:35.724856 3813 log.cc:1196] T b2bc9547219d46ce8dcfbb2558e3f32a P be9437917ecf496e852fe0f5bb10bec8: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-1/wal/wals/b2bc9547219d46ce8dcfbb2558e3f32a
I20260812 03:08:35.725183 3813 ts_tablet_manager.cc:1939] T b2bc9547219d46ce8dcfbb2558e3f32a P be9437917ecf496e852fe0f5bb10bec8: Deleting consensus metadata
I20260812 03:08:35.726044 3175 catalog_manager.cc:4841] TS be9437917ecf496e852fe0f5bb10bec8 (127.26.250.130:45369): tablet b2bc9547219d46ce8dcfbb2558e3f32a (table test-table [id=065d3f06174f4d82b09008770aa42c30]) successfully deleted
Aug 12 03:08:35 dist-test-slave-zpfg krb5kdc[3145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 03:08:35 dist-test-slave-zpfg krb5kdc[3145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:35 dist-test-slave-zpfg krb5kdc[3145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:08:35.754933 3776 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:35.747569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47344 (local address 127.26.250.190:32913)
0812 03:08:35.747741 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:35.747746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:35.747957 (+ 211us) server_negotiation.cc:407] Connection header received
0812 03:08:35.748134 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:35.748138 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:35.748197 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:35.748291 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:35.749102 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.749705 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.750866 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.751185 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.751251 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:35.754821 (+ 3570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47344: BlockingRecv error: recv got EOF from 127.0.0.1:47344 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 03:08:35 dist-test-slave-zpfg krb5kdc[3145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM
I20260812 03:08:35.766690 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:35.757309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47356 (local address 127.26.250.190:32913)
0812 03:08:35.757429 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:35.757435 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:35.757634 (+ 199us) server_negotiation.cc:407] Connection header received
0812 03:08:35.757787 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:35.757790 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:35.757846 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:35.757933 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:35.758672 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.759224 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.760115 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.760432 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.760566 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:35.763222 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:35.763231 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:35.763233 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:35.763259 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:35.764777 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:35.765217 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:35.765221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:35.765223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:35.765318 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:35.765610 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:35.765615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:35.765617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:35.765809 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:35.765959 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:35.766212 (+ 253us) server_negotiation.cc:299] Negotiation successful
0812 03:08:35.766380 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 03:08:35.769487 3189 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:08:35.769985 3189 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:08:35.775597 3637 tablet_service.cc:1467] Processing CreateTablet for tablet 3a77e94ebcec47a986e3a88ab38325b8 (DEFAULT_TABLE table=test-table [id=50b21841a9424f949f59c7ffcf7519a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:35.775640 3337 tablet_service.cc:1467] Processing CreateTablet for tablet 3a77e94ebcec47a986e3a88ab38325b8 (DEFAULT_TABLE table=test-table [id=50b21841a9424f949f59c7ffcf7519a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:35.775934 3337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a77e94ebcec47a986e3a88ab38325b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:35.775934 3637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a77e94ebcec47a986e3a88ab38325b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:35.778703 3489 tablet_service.cc:1467] Processing CreateTablet for tablet 3a77e94ebcec47a986e3a88ab38325b8 (DEFAULT_TABLE table=test-table [id=50b21841a9424f949f59c7ffcf7519a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:35.778903 3737 tablet_bootstrap.cc:492] T 3a77e94ebcec47a986e3a88ab38325b8 P 32acdec1a00646eca00082ba4095be15: Bootstrap starting.
I20260812 03:08:35.779029 3489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a77e94ebcec47a986e3a88ab38325b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:35.779872 3737 tablet_bootstrap.cc:654] T 3a77e94ebcec47a986e3a88ab38325b8 P 32acdec1a00646eca00082ba4095be15: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:35.781507 3736 tablet_bootstrap.cc:492] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8: Bootstrap starting.
I20260812 03:08:35.782552 3736 tablet_bootstrap.cc:654] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:35.782929 3735 tablet_bootstrap.cc:492] T 3a77e94ebcec47a986e3a88ab38325b8 P 6801eb5071dc43bcbd45983831f97ce7: Bootstrap starting.
I20260812 03:08:35.784057 3735 tablet_bootstrap.cc:654] T 3a77e94ebcec47a986e3a88ab38325b8 P 6801eb5071dc43bcbd45983831f97ce7: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:35.784111 3736 tablet_bootstrap.cc:492] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8: No bootstrap required, opened a new log
I20260812 03:08:35.784196 3736 ts_tablet_manager.cc:1397] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 03:08:35.784686 3736 raft_consensus.cc:348] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.784814 3736 raft_consensus.cc:374] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:35.784847 3736 raft_consensus.cc:729] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9437917ecf496e852fe0f5bb10bec8, State: Initialized, Role: FOLLOWER
I20260812 03:08:35.784986 3736 consensus_queue.cc:260] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.785283 3736 ts_tablet_manager.cc:1428] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:08:35.787489 3735 tablet_bootstrap.cc:492] T 3a77e94ebcec47a986e3a88ab38325b8 P 6801eb5071dc43bcbd45983831f97ce7: No bootstrap required, opened a new log
I20260812 03:08:35.787614 3735 ts_tablet_manager.cc:1397] T 3a77e94ebcec47a986e3a88ab38325b8 P 6801eb5071dc43bcbd45983831f97ce7: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260812 03:08:35.788113 3735 raft_consensus.cc:348] T 3a77e94ebcec47a986e3a88ab38325b8 P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.788242 3735 raft_consensus.cc:374] T 3a77e94ebcec47a986e3a88ab38325b8 P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:35.788283 3735 raft_consensus.cc:729] T 3a77e94ebcec47a986e3a88ab38325b8 P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6801eb5071dc43bcbd45983831f97ce7, State: Initialized, Role: FOLLOWER
I20260812 03:08:35.788416 3735 consensus_queue.cc:260] T 3a77e94ebcec47a986e3a88ab38325b8 P 6801eb5071dc43bcbd45983831f97ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.788687 3735 ts_tablet_manager.cc:1428] T 3a77e94ebcec47a986e3a88ab38325b8 P 6801eb5071dc43bcbd45983831f97ce7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:08:35.791440 3737 tablet_bootstrap.cc:492] T 3a77e94ebcec47a986e3a88ab38325b8 P 32acdec1a00646eca00082ba4095be15: No bootstrap required, opened a new log
I20260812 03:08:35.791576 3737 ts_tablet_manager.cc:1397] T 3a77e94ebcec47a986e3a88ab38325b8 P 32acdec1a00646eca00082ba4095be15: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20260812 03:08:35.792034 3737 raft_consensus.cc:348] T 3a77e94ebcec47a986e3a88ab38325b8 P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.792155 3737 raft_consensus.cc:374] T 3a77e94ebcec47a986e3a88ab38325b8 P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:35.792187 3737 raft_consensus.cc:729] T 3a77e94ebcec47a986e3a88ab38325b8 P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32acdec1a00646eca00082ba4095be15, State: Initialized, Role: FOLLOWER
I20260812 03:08:35.792366 3737 consensus_queue.cc:260] T 3a77e94ebcec47a986e3a88ab38325b8 P 32acdec1a00646eca00082ba4095be15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.792670 3737 ts_tablet_manager.cc:1428] T 3a77e94ebcec47a986e3a88ab38325b8 P 32acdec1a00646eca00082ba4095be15: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:08:35.954874 3803 raft_consensus.cc:482] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:08:35.955080 3803 raft_consensus.cc:504] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.955703 3803 leader_election.cc:288] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6801eb5071dc43bcbd45983831f97ce7 (127.26.250.129:39723), 32acdec1a00646eca00082ba4095be15 (127.26.250.131:40591)
I20260812 03:08:35.956261 3359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a77e94ebcec47a986e3a88ab38325b8" candidate_uuid: "be9437917ecf496e852fe0f5bb10bec8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6801eb5071dc43bcbd45983831f97ce7" is_pre_election: true
I20260812 03:08:35.956260 3658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a77e94ebcec47a986e3a88ab38325b8" candidate_uuid: "be9437917ecf496e852fe0f5bb10bec8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32acdec1a00646eca00082ba4095be15" is_pre_election: true
I20260812 03:08:35.956439 3359 raft_consensus.cc:2454] T 3a77e94ebcec47a986e3a88ab38325b8 P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9437917ecf496e852fe0f5bb10bec8 in term 0.
I20260812 03:08:35.956481 3658 raft_consensus.cc:2454] T 3a77e94ebcec47a986e3a88ab38325b8 P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be9437917ecf496e852fe0f5bb10bec8 in term 0.
I20260812 03:08:35.956842 3437 leader_election.cc:302] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6801eb5071dc43bcbd45983831f97ce7, be9437917ecf496e852fe0f5bb10bec8; no voters:
I20260812 03:08:35.957029 3803 raft_consensus.cc:2781] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:08:35.957091 3803 raft_consensus.cc:482] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:08:35.957142 3803 raft_consensus.cc:3037] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:35.958401 3803 raft_consensus.cc:504] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.958747 3803 leader_election.cc:288] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8 [CANDIDATE]: Term 1 election: Requested vote from peers 6801eb5071dc43bcbd45983831f97ce7 (127.26.250.129:39723), 32acdec1a00646eca00082ba4095be15 (127.26.250.131:40591)
I20260812 03:08:35.959126 3359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a77e94ebcec47a986e3a88ab38325b8" candidate_uuid: "be9437917ecf496e852fe0f5bb10bec8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6801eb5071dc43bcbd45983831f97ce7"
I20260812 03:08:35.959265 3359 raft_consensus.cc:3037] T 3a77e94ebcec47a986e3a88ab38325b8 P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:35.959220 3658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a77e94ebcec47a986e3a88ab38325b8" candidate_uuid: "be9437917ecf496e852fe0f5bb10bec8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32acdec1a00646eca00082ba4095be15"
I20260812 03:08:35.959379 3658 raft_consensus.cc:3037] T 3a77e94ebcec47a986e3a88ab38325b8 P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:35.960522 3359 raft_consensus.cc:2454] T 3a77e94ebcec47a986e3a88ab38325b8 P 6801eb5071dc43bcbd45983831f97ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9437917ecf496e852fe0f5bb10bec8 in term 1.
I20260812 03:08:35.960565 3658 raft_consensus.cc:2454] T 3a77e94ebcec47a986e3a88ab38325b8 P 32acdec1a00646eca00082ba4095be15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be9437917ecf496e852fe0f5bb10bec8 in term 1.
I20260812 03:08:35.960886 3437 leader_election.cc:302] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6801eb5071dc43bcbd45983831f97ce7, be9437917ecf496e852fe0f5bb10bec8; no voters:
I20260812 03:08:35.961066 3803 raft_consensus.cc:2781] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:35.961273 3803 raft_consensus.cc:686] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8 [term 1 LEADER]: Becoming Leader. State: Replica: be9437917ecf496e852fe0f5bb10bec8, State: Running, Role: LEADER
I20260812 03:08:35.961438 3803 consensus_queue.cc:237] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:35.964339 3189 catalog_manager.cc:5495] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8 reported cstate change: term changed from 0 to 1, leader changed from <none> to be9437917ecf496e852fe0f5bb10bec8 (127.26.250.130). New cstate: current_term: 1 leader_uuid: "be9437917ecf496e852fe0f5bb10bec8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } health_report { overall_health: UNKNOWN } } }
I20260812 03:08:35.982303 3729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:35.979070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55760 (local address 127.26.250.130:45369)
0812 03:08:35.979211 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:35.979216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:35.979231 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:08:35.979355 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:35.979358 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:35.979416 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:35.979502 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:08:35.979767 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.980291 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.981186 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:35.981422 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:35.981549 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:35.981825 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:08:35.981913 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:08:35.981993 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:35.982096 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 03:08:35.982153 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:08:35.984896 3658 raft_consensus.cc:1261] T 3a77e94ebcec47a986e3a88ab38325b8 P 32acdec1a00646eca00082ba4095be15 [term 1 FOLLOWER]: Refusing update from remote peer be9437917ecf496e852fe0f5bb10bec8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:35.984944 3359 raft_consensus.cc:1261] T 3a77e94ebcec47a986e3a88ab38325b8 P 6801eb5071dc43bcbd45983831f97ce7 [term 1 FOLLOWER]: Refusing update from remote peer be9437917ecf496e852fe0f5bb10bec8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:35.985378 3803 consensus_queue.cc:1035] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:35.985543 3741 consensus_queue.cc:1035] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:35.997287 3189 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:35.997500 3189 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:36.000052 3189 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf: Sending DeleteTablet for 3 replicas of tablet 3a77e94ebcec47a986e3a88ab38325b8
I20260812 03:08:36.000777 3337 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a77e94ebcec47a986e3a88ab38325b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:35 UTC) from {username='oryx', principal='oryx/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:38076
I20260812 03:08:36.000901 3637 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a77e94ebcec47a986e3a88ab38325b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:35 UTC) from {username='oryx', principal='oryx/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:52738
I20260812 03:08:36.000994 3812 tablet_replica.cc:330] T 3a77e94ebcec47a986e3a88ab38325b8 P 6801eb5071dc43bcbd45983831f97ce7: stopping tablet replica
I20260812 03:08:36.001101 3811 tablet_replica.cc:330] T 3a77e94ebcec47a986e3a88ab38325b8 P 32acdec1a00646eca00082ba4095be15: stopping tablet replica
I20260812 03:08:36.001132 3812 raft_consensus.cc:2229] T 3a77e94ebcec47a986e3a88ab38325b8 P 6801eb5071dc43bcbd45983831f97ce7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:36.001240 3811 raft_consensus.cc:2229] T 3a77e94ebcec47a986e3a88ab38325b8 P 32acdec1a00646eca00082ba4095be15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:36.001287 3812 raft_consensus.cc:2258] T 3a77e94ebcec47a986e3a88ab38325b8 P 6801eb5071dc43bcbd45983831f97ce7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:36.001401 3811 raft_consensus.cc:2258] T 3a77e94ebcec47a986e3a88ab38325b8 P 32acdec1a00646eca00082ba4095be15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:36.001605 3190 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:08:36.002075 3190 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:08:36.002172 3812 ts_tablet_manager.cc:1905] T 3a77e94ebcec47a986e3a88ab38325b8 P 6801eb5071dc43bcbd45983831f97ce7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:36.002295 3811 ts_tablet_manager.cc:1905] T 3a77e94ebcec47a986e3a88ab38325b8 P 32acdec1a00646eca00082ba4095be15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:36.004842 3811 ts_tablet_manager.cc:1918] T 3a77e94ebcec47a986e3a88ab38325b8 P 32acdec1a00646eca00082ba4095be15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:08:36.004912 3811 log.cc:1196] T 3a77e94ebcec47a986e3a88ab38325b8 P 32acdec1a00646eca00082ba4095be15: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/3a77e94ebcec47a986e3a88ab38325b8
I20260812 03:08:36.005008 3812 ts_tablet_manager.cc:1918] T 3a77e94ebcec47a986e3a88ab38325b8 P 6801eb5071dc43bcbd45983831f97ce7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:08:36.005086 3812 log.cc:1196] T 3a77e94ebcec47a986e3a88ab38325b8 P 6801eb5071dc43bcbd45983831f97ce7: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-0/wal/wals/3a77e94ebcec47a986e3a88ab38325b8
I20260812 03:08:36.005103 3811 ts_tablet_manager.cc:1939] T 3a77e94ebcec47a986e3a88ab38325b8 P 32acdec1a00646eca00082ba4095be15: Deleting consensus metadata
I20260812 03:08:36.005404 3812 ts_tablet_manager.cc:1939] T 3a77e94ebcec47a986e3a88ab38325b8 P 6801eb5071dc43bcbd45983831f97ce7: Deleting consensus metadata
I20260812 03:08:36.006453 3175 catalog_manager.cc:4841] TS 6801eb5071dc43bcbd45983831f97ce7 (127.26.250.129:39723): tablet 3a77e94ebcec47a986e3a88ab38325b8 (table test-table [id=50b21841a9424f949f59c7ffcf7519a5]) successfully deleted
I20260812 03:08:36.007547 3175 catalog_manager.cc:4841] TS 32acdec1a00646eca00082ba4095be15 (127.26.250.131:40591): tablet 3a77e94ebcec47a986e3a88ab38325b8 (table test-table [id=50b21841a9424f949f59c7ffcf7519a5]) successfully deleted
I20260812 03:08:36.007965 3489 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a77e94ebcec47a986e3a88ab38325b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:35 UTC) from {username='oryx', principal='oryx/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:55738
I20260812 03:08:36.008169 3813 tablet_replica.cc:330] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8: stopping tablet replica
I20260812 03:08:36.008304 3813 raft_consensus.cc:2229] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:08:36.008692 3813 raft_consensus.cc:2258] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:36.009486 3813 ts_tablet_manager.cc:1905] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:36.009764 3337 tablet_service.cc:1467] Processing CreateTablet for tablet f12dafc8e5f44cb0af1fd35bd107ffaf (DEFAULT_TABLE table=test-table [id=46c83706599f4df9a3335b4cb83a996d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:36.010047 3337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f12dafc8e5f44cb0af1fd35bd107ffaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:36.012235 3813 ts_tablet_manager.cc:1918] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:08:36.012311 3813 log.cc:1196] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-1/wal/wals/3a77e94ebcec47a986e3a88ab38325b8
I20260812 03:08:36.012571 3813 ts_tablet_manager.cc:1939] T 3a77e94ebcec47a986e3a88ab38325b8 P be9437917ecf496e852fe0f5bb10bec8: Deleting consensus metadata
I20260812 03:08:36.012704 3735 tablet_bootstrap.cc:492] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7: Bootstrap starting.
I20260812 03:08:36.013340 3175 catalog_manager.cc:4841] TS be9437917ecf496e852fe0f5bb10bec8 (127.26.250.130:45369): tablet 3a77e94ebcec47a986e3a88ab38325b8 (table test-table [id=50b21841a9424f949f59c7ffcf7519a5]) successfully deleted
I20260812 03:08:36.013401 3489 tablet_service.cc:1467] Processing CreateTablet for tablet f12dafc8e5f44cb0af1fd35bd107ffaf (DEFAULT_TABLE table=test-table [id=46c83706599f4df9a3335b4cb83a996d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:36.013574 3735 tablet_bootstrap.cc:654] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:36.013666 3489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f12dafc8e5f44cb0af1fd35bd107ffaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:36.014964 3637 tablet_service.cc:1467] Processing CreateTablet for tablet f12dafc8e5f44cb0af1fd35bd107ffaf (DEFAULT_TABLE table=test-table [id=46c83706599f4df9a3335b4cb83a996d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:08:36.015270 3637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f12dafc8e5f44cb0af1fd35bd107ffaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:36.015538 3735 tablet_bootstrap.cc:492] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7: No bootstrap required, opened a new log
I20260812 03:08:36.015621 3735 ts_tablet_manager.cc:1397] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 03:08:36.016074 3735 raft_consensus.cc:348] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:36.016208 3735 raft_consensus.cc:374] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:36.016242 3735 raft_consensus.cc:729] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6801eb5071dc43bcbd45983831f97ce7, State: Initialized, Role: FOLLOWER
I20260812 03:08:36.016275 3736 tablet_bootstrap.cc:492] T f12dafc8e5f44cb0af1fd35bd107ffaf P be9437917ecf496e852fe0f5bb10bec8: Bootstrap starting.
I20260812 03:08:36.016377 3735 consensus_queue.cc:260] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:36.016664 3735 ts_tablet_manager.cc:1428] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:08:36.017201 3736 tablet_bootstrap.cc:654] T f12dafc8e5f44cb0af1fd35bd107ffaf P be9437917ecf496e852fe0f5bb10bec8: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:36.017865 3737 tablet_bootstrap.cc:492] T f12dafc8e5f44cb0af1fd35bd107ffaf P 32acdec1a00646eca00082ba4095be15: Bootstrap starting.
I20260812 03:08:36.018838 3737 tablet_bootstrap.cc:654] T f12dafc8e5f44cb0af1fd35bd107ffaf P 32acdec1a00646eca00082ba4095be15: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:36.018922 3736 tablet_bootstrap.cc:492] T f12dafc8e5f44cb0af1fd35bd107ffaf P be9437917ecf496e852fe0f5bb10bec8: No bootstrap required, opened a new log
I20260812 03:08:36.018998 3736 ts_tablet_manager.cc:1397] T f12dafc8e5f44cb0af1fd35bd107ffaf P be9437917ecf496e852fe0f5bb10bec8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260812 03:08:36.019436 3736 raft_consensus.cc:348] T f12dafc8e5f44cb0af1fd35bd107ffaf P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:36.019574 3736 raft_consensus.cc:374] T f12dafc8e5f44cb0af1fd35bd107ffaf P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:36.019616 3736 raft_consensus.cc:729] T f12dafc8e5f44cb0af1fd35bd107ffaf P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be9437917ecf496e852fe0f5bb10bec8, State: Initialized, Role: FOLLOWER
I20260812 03:08:36.019749 3736 consensus_queue.cc:260] T f12dafc8e5f44cb0af1fd35bd107ffaf P be9437917ecf496e852fe0f5bb10bec8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:36.019946 3737 tablet_bootstrap.cc:492] T f12dafc8e5f44cb0af1fd35bd107ffaf P 32acdec1a00646eca00082ba4095be15: No bootstrap required, opened a new log
I20260812 03:08:36.020022 3737 ts_tablet_manager.cc:1397] T f12dafc8e5f44cb0af1fd35bd107ffaf P 32acdec1a00646eca00082ba4095be15: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 03:08:36.020052 3736 ts_tablet_manager.cc:1428] T f12dafc8e5f44cb0af1fd35bd107ffaf P be9437917ecf496e852fe0f5bb10bec8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 03:08:36.020458 3737 raft_consensus.cc:348] T f12dafc8e5f44cb0af1fd35bd107ffaf P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:36.020576 3737 raft_consensus.cc:374] T f12dafc8e5f44cb0af1fd35bd107ffaf P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:36.020617 3737 raft_consensus.cc:729] T f12dafc8e5f44cb0af1fd35bd107ffaf P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32acdec1a00646eca00082ba4095be15, State: Initialized, Role: FOLLOWER
I20260812 03:08:36.020741 3737 consensus_queue.cc:260] T f12dafc8e5f44cb0af1fd35bd107ffaf P 32acdec1a00646eca00082ba4095be15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:36.021020 3737 ts_tablet_manager.cc:1428] T f12dafc8e5f44cb0af1fd35bd107ffaf P 32acdec1a00646eca00082ba4095be15: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:08:36.189186 3743 raft_consensus.cc:482] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:08:36.189352 3743 raft_consensus.cc:504] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:36.189900 3743 leader_election.cc:288] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be9437917ecf496e852fe0f5bb10bec8 (127.26.250.130:45369), 32acdec1a00646eca00082ba4095be15 (127.26.250.131:40591)
I20260812 03:08:36.190501 3658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f12dafc8e5f44cb0af1fd35bd107ffaf" candidate_uuid: "6801eb5071dc43bcbd45983831f97ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32acdec1a00646eca00082ba4095be15" is_pre_election: true
I20260812 03:08:36.190502 3512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f12dafc8e5f44cb0af1fd35bd107ffaf" candidate_uuid: "6801eb5071dc43bcbd45983831f97ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9437917ecf496e852fe0f5bb10bec8" is_pre_election: true
I20260812 03:08:36.190668 3512 raft_consensus.cc:2454] T f12dafc8e5f44cb0af1fd35bd107ffaf P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6801eb5071dc43bcbd45983831f97ce7 in term 0.
I20260812 03:08:36.190668 3658 raft_consensus.cc:2454] T f12dafc8e5f44cb0af1fd35bd107ffaf P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6801eb5071dc43bcbd45983831f97ce7 in term 0.
I20260812 03:08:36.191026 3286 leader_election.cc:302] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6801eb5071dc43bcbd45983831f97ce7, be9437917ecf496e852fe0f5bb10bec8; no voters:
I20260812 03:08:36.191211 3743 raft_consensus.cc:2781] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:08:36.191275 3743 raft_consensus.cc:482] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:08:36.191314 3743 raft_consensus.cc:3037] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:36.192363 3743 raft_consensus.cc:504] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:36.192706 3743 leader_election.cc:288] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7 [CANDIDATE]: Term 1 election: Requested vote from peers be9437917ecf496e852fe0f5bb10bec8 (127.26.250.130:45369), 32acdec1a00646eca00082ba4095be15 (127.26.250.131:40591)
I20260812 03:08:36.193125 3512 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f12dafc8e5f44cb0af1fd35bd107ffaf" candidate_uuid: "6801eb5071dc43bcbd45983831f97ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be9437917ecf496e852fe0f5bb10bec8"
I20260812 03:08:36.193135 3658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f12dafc8e5f44cb0af1fd35bd107ffaf" candidate_uuid: "6801eb5071dc43bcbd45983831f97ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32acdec1a00646eca00082ba4095be15"
I20260812 03:08:36.193267 3658 raft_consensus.cc:3037] T f12dafc8e5f44cb0af1fd35bd107ffaf P 32acdec1a00646eca00082ba4095be15 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:36.193267 3512 raft_consensus.cc:3037] T f12dafc8e5f44cb0af1fd35bd107ffaf P be9437917ecf496e852fe0f5bb10bec8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:36.194526 3512 raft_consensus.cc:2454] T f12dafc8e5f44cb0af1fd35bd107ffaf P be9437917ecf496e852fe0f5bb10bec8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6801eb5071dc43bcbd45983831f97ce7 in term 1.
I20260812 03:08:36.194537 3658 raft_consensus.cc:2454] T f12dafc8e5f44cb0af1fd35bd107ffaf P 32acdec1a00646eca00082ba4095be15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6801eb5071dc43bcbd45983831f97ce7 in term 1.
I20260812 03:08:36.194860 3286 leader_election.cc:302] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32acdec1a00646eca00082ba4095be15, 6801eb5071dc43bcbd45983831f97ce7; no voters:
I20260812 03:08:36.195075 3743 raft_consensus.cc:2781] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:36.195186 3743 raft_consensus.cc:686] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7 [term 1 LEADER]: Becoming Leader. State: Replica: 6801eb5071dc43bcbd45983831f97ce7, State: Running, Role: LEADER
I20260812 03:08:36.195356 3743 consensus_queue.cc:237] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } }
I20260812 03:08:36.197204 3189 catalog_manager.cc:5495] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6801eb5071dc43bcbd45983831f97ce7 (127.26.250.129). New cstate: current_term: 1 leader_uuid: "6801eb5071dc43bcbd45983831f97ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6801eb5071dc43bcbd45983831f97ce7" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 39723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 } health_report { overall_health: UNKNOWN } } }
I20260812 03:08:36.218727 3838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:36.215313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38090 (local address 127.26.250.129:39723)
0812 03:08:36.215616 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:36.215619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:36.215630 (+ 11us) server_negotiation.cc:407] Connection header received
0812 03:08:36.215668 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:36.215670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:36.215710 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:36.215807 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:08:36.216061 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:36.216726 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:36.217595 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:36.217837 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:36.217947 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:36.218215 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:08:36.218297 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:08:36.218400 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:36.218502 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 03:08:36.218556 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":126,"threads_started":1}
I20260812 03:08:36.225835 3658 raft_consensus.cc:1261] T f12dafc8e5f44cb0af1fd35bd107ffaf P 32acdec1a00646eca00082ba4095be15 [term 1 FOLLOWER]: Refusing update from remote peer 6801eb5071dc43bcbd45983831f97ce7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:36.225839 3512 raft_consensus.cc:1261] T f12dafc8e5f44cb0af1fd35bd107ffaf P be9437917ecf496e852fe0f5bb10bec8 [term 1 FOLLOWER]: Refusing update from remote peer 6801eb5071dc43bcbd45983831f97ce7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:08:36.226276 3743 consensus_queue.cc:1035] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32acdec1a00646eca00082ba4095be15" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 40591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:36.226512 3769 consensus_queue.cc:1035] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be9437917ecf496e852fe0f5bb10bec8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 45369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:08:36.260314 3189 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:36.260489 3189 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:08:36.262522 3189 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 712dad23c9c343f29dbc9bc3fb5d04cf: Sending DeleteTablet for 3 replicas of tablet f12dafc8e5f44cb0af1fd35bd107ffaf
I20260812 03:08:36.263156 3337 tablet_service.cc:1514] Processing DeleteTablet for tablet f12dafc8e5f44cb0af1fd35bd107ffaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:36 UTC) from {username='oryx', principal='oryx/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:38076
I20260812 03:08:36.263329 3489 tablet_service.cc:1514] Processing DeleteTablet for tablet f12dafc8e5f44cb0af1fd35bd107ffaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:36 UTC) from {username='oryx', principal='oryx/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:55738
I20260812 03:08:36.263399 3812 tablet_replica.cc:330] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7: stopping tablet replica
I20260812 03:08:36.263368 3637 tablet_service.cc:1514] Processing DeleteTablet for tablet f12dafc8e5f44cb0af1fd35bd107ffaf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:08:36 UTC) from {username='oryx', principal='oryx/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:52738
I20260812 03:08:36.263512 3813 tablet_replica.cc:330] T f12dafc8e5f44cb0af1fd35bd107ffaf P be9437917ecf496e852fe0f5bb10bec8: stopping tablet replica
I20260812 03:08:36.263535 3812 raft_consensus.cc:2229] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:08:36.263603 3811 tablet_replica.cc:330] T f12dafc8e5f44cb0af1fd35bd107ffaf P 32acdec1a00646eca00082ba4095be15: stopping tablet replica
I20260812 03:08:36.263620 3813 raft_consensus.cc:2229] T f12dafc8e5f44cb0af1fd35bd107ffaf P be9437917ecf496e852fe0f5bb10bec8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:36.263717 3811 raft_consensus.cc:2229] T f12dafc8e5f44cb0af1fd35bd107ffaf P 32acdec1a00646eca00082ba4095be15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:08:36.263720 3812 raft_consensus.cc:2258] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:36.263728 3813 raft_consensus.cc:2258] T f12dafc8e5f44cb0af1fd35bd107ffaf P be9437917ecf496e852fe0f5bb10bec8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:36.263815 3811 raft_consensus.cc:2258] T f12dafc8e5f44cb0af1fd35bd107ffaf P 32acdec1a00646eca00082ba4095be15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:08:36.264555 3812 ts_tablet_manager.cc:1905] T f12dafc8e5f44cb0af1fd35bd107ffaf P 6801eb5071dc43bcbd45983831f97ce7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:36.264693 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 3268
I20260812 03:08:36.264700 3813 ts_tablet_manager.cc:1905] T f12dafc8e5f44cb0af1fd35bd107ffaf P be9437917ecf496e852fe0f5bb10bec8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:36.264698 3811 ts_tablet_manager.cc:1905] T f12dafc8e5f44cb0af1fd35bd107ffaf P 32acdec1a00646eca00082ba4095be15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:08:36.267592 3813 ts_tablet_manager.cc:1918] T f12dafc8e5f44cb0af1fd35bd107ffaf P be9437917ecf496e852fe0f5bb10bec8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:08:36.267580 3811 ts_tablet_manager.cc:1918] T f12dafc8e5f44cb0af1fd35bd107ffaf P 32acdec1a00646eca00082ba4095be15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 03:08:36.267673 3813 log.cc:1196] T f12dafc8e5f44cb0af1fd35bd107ffaf P be9437917ecf496e852fe0f5bb10bec8: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-1/wal/wals/f12dafc8e5f44cb0af1fd35bd107ffaf
I20260812 03:08:36.267673 3811 log.cc:1196] T f12dafc8e5f44cb0af1fd35bd107ffaf P 32acdec1a00646eca00082ba4095be15: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/f12dafc8e5f44cb0af1fd35bd107ffaf
I20260812 03:08:36.267959 3811 ts_tablet_manager.cc:1939] T f12dafc8e5f44cb0af1fd35bd107ffaf P 32acdec1a00646eca00082ba4095be15: Deleting consensus metadata
I20260812 03:08:36.267958 3813 ts_tablet_manager.cc:1939] T f12dafc8e5f44cb0af1fd35bd107ffaf P be9437917ecf496e852fe0f5bb10bec8: Deleting consensus metadata
I20260812 03:08:36.268955 3175 catalog_manager.cc:4841] TS be9437917ecf496e852fe0f5bb10bec8 (127.26.250.130:45369): tablet f12dafc8e5f44cb0af1fd35bd107ffaf (table test-table [id=46c83706599f4df9a3335b4cb83a996d]) successfully deleted
I20260812 03:08:36.269486 3175 catalog_manager.cc:4841] TS 32acdec1a00646eca00082ba4095be15 (127.26.250.131:40591): tablet f12dafc8e5f44cb0af1fd35bd107ffaf (table test-table [id=46c83706599f4df9a3335b4cb83a996d]) successfully deleted
I20260812 03:08:36.277688 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 3419
W20260812 03:08:36.277663 3175 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.250.129:39723 (error 108)
I20260812 03:08:36.281283 3846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:36.280634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:39723 (local address 127.0.0.1:38096)
0812 03:08:36.280980 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:36.281059 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.129:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":91,"threads_started":1}
W20260812 03:08:36.281814 3175 catalog_manager.cc:4584] TS 6801eb5071dc43bcbd45983831f97ce7 (127.26.250.129:39723): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f12dafc8e5f44cb0af1fd35bd107ffaf: Network error: Client connection negotiation failed: client connection to 127.26.250.129:39723: connect: Connection refused (error 111)
I20260812 03:08:36.288650 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 3567
I20260812 03:08:36.298483 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 3159
2026-08-12T03:08:36Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3857 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:08:36 dist-test-slave-zpfg krb5kdc[3849](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:08:36 dist-test-slave-zpfg krb5kdc[3849](info): set up 2 sockets
Aug 12 03:08:36 dist-test-slave-zpfg krb5kdc[3849](info): commencing operation
krb5kdc: starting...
W20260812 03:08:37.416769 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:08:37 dist-test-slave-zpfg krb5kdc[3849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:08:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:08:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:08:37.558917 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:35075
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:35075
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:37.676820 3862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:37.677112 3862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:37.677155 3862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:37.681213 3862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:08:37.681293 3862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:37.681314 3862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:37.681331 3862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:08:37.681347 3862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:08:37.686839 3862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:35075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:35075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:37.687928 3862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:37.688931 3862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:37.697989 3872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:37.698028 3870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:37.698257 3862 server_base.cc:1034] running on GCE node
W20260812 03:08:37.697994 3867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:37.698940 3862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:37.700155 3862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:37.701357 3862 hybrid_clock.cc:648] HybridClock initialized: now 1786504117701309 us; error 64 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T03:08:37Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1404 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:08:37 dist-test-slave-zpfg krb5kdc[3877](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:08:37 dist-test-slave-zpfg krb5kdc[3877](info): set up 2 sockets
Aug 12 03:08:37 dist-test-slave-zpfg krb5kdc[3877](info): commencing operation
krb5kdc: starting...
W20260812 03:08:38.804632 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:08:38 dist-test-slave-zpfg krb5kdc[3877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:08:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:08:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:08:38.946038 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:34905
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:36383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:34905
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:39.086768 3891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:39.087116 3891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:39.087208 3891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:39.091533 3891 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:08:39.091642 3891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:39.091678 3891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:39.091732 3891 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:08:39.091771 3891 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:08:39.097523 3891 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:36383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:34905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:34905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:39.098867 3891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:39.099997 3891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:39.109773 3899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:39.109798 3896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:39.109773 3897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:39.109828 3891 server_base.cc:1034] running on GCE node
I20260812 03:08:39.110772 3891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:39.112053 3891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:39.113250 3891 hybrid_clock.cc:648] HybridClock initialized: now 1786504119113233 us; error 32 us; skew 500 ppm
Aug 12 03:08:39 dist-test-slave-zpfg krb5kdc[3877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504119, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:39.116420 3891 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:08:39.117386 3891 webserver.cc:460] Webserver started at http://127.26.250.190:40381/ using document root <none> and password file <none>
I20260812 03:08:39.117937 3891 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:39.117998 3891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:39.118173 3891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:39.120347 3891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "a0eae0e089734f02bb34300e0dfe9bad"
format_stamp: "Formatted at 2026-08-12 03:08:39 on dist-test-slave-zpfg"
server_key: "ebd02d7d9149f5d6a80fdf43ae9aa10b"
server_key_iv: "03d65d0f18160005b357f810b6dc7e60"
server_key_version: "encryptionkey@0"
I20260812 03:08:39.120891 3891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "a0eae0e089734f02bb34300e0dfe9bad"
format_stamp: "Formatted at 2026-08-12 03:08:39 on dist-test-slave-zpfg"
server_key: "ebd02d7d9149f5d6a80fdf43ae9aa10b"
server_key_iv: "03d65d0f18160005b357f810b6dc7e60"
server_key_version: "encryptionkey@0"
I20260812 03:08:39.124876 3891 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:08:39.127522 3906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:39.128952 3891 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:08:39.129149 3891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "a0eae0e089734f02bb34300e0dfe9bad"
format_stamp: "Formatted at 2026-08-12 03:08:39 on dist-test-slave-zpfg"
server_key: "ebd02d7d9149f5d6a80fdf43ae9aa10b"
server_key_iv: "03d65d0f18160005b357f810b6dc7e60"
server_key_version: "encryptionkey@0"
I20260812 03:08:39.129284 3891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:39.148870 3891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:39.152068 3891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:39.152293 3891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:39.160213 3891 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:34905
I20260812 03:08:39.160241 3960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:34905 every 8 connection(s)
I20260812 03:08:39.161449 3891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:08:39.164283 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 3891
I20260812 03:08:39.164433 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:08:39.164759 27626 external_mini_cluster.cc:1428] Setting key c1fa0757bb63dffc8225f56984b08b21
I20260812 03:08:39.165315 3961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:39.171844 3961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad: Bootstrap starting.
Aug 12 03:08:39 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:39.174455 3961 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:39.175195 3961 log.cc:824] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:39.177693 3961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad: No bootstrap required, opened a new log
I20260812 03:08:39.180459 3964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:39.166381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36514 (local address 127.26.250.190:34905)
0812 03:08:39.166914 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:39.166931 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:39.166970 (+ 39us) server_negotiation.cc:407] Connection header received
0812 03:08:39.167635 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:39.167652 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:39.167994 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:39.168358 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:39.169204 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:39.170251 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:39.171158 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:39.171658 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:39.171848 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:39.174830 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:39.174836 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:39.174850 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:39.174879 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:39.177022 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:39.177488 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:39.177494 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:39.177499 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:39.177651 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:39.177977 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:39.177980 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:39.177982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:39.178533 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:39.178751 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:39.179041 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 03:08:39.179352 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":141,"threads_started":1}
I20260812 03:08:39.182502 3961 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0eae0e089734f02bb34300e0dfe9bad" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 34905 } }
I20260812 03:08:39.182780 3961 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:39.182875 3961 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0eae0e089734f02bb34300e0dfe9bad, State: Initialized, Role: FOLLOWER
I20260812 03:08:39.183318 3961 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0eae0e089734f02bb34300e0dfe9bad" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 34905 } }
I20260812 03:08:39.183459 3961 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:08:39.183499 3961 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:08:39.183574 3961 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:39.184495 3961 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0eae0e089734f02bb34300e0dfe9bad" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 34905 } }
I20260812 03:08:39.184804 3961 leader_election.cc:302] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0eae0e089734f02bb34300e0dfe9bad; no voters:
I20260812 03:08:39.185110 3961 leader_election.cc:288] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:08:39.185276 3966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:39.185648 3966 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [term 1 LEADER]: Becoming Leader. State: Replica: a0eae0e089734f02bb34300e0dfe9bad, State: Running, Role: LEADER
I20260812 03:08:39.186051 3966 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0eae0e089734f02bb34300e0dfe9bad" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 34905 } }
I20260812 03:08:39.186503 3961 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:08:39.187871 3968 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0eae0e089734f02bb34300e0dfe9bad. Latest consensus state: current_term: 1 leader_uuid: "a0eae0e089734f02bb34300e0dfe9bad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0eae0e089734f02bb34300e0dfe9bad" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 34905 } } }
I20260812 03:08:39.188011 3968 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:39.188356 3967 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0eae0e089734f02bb34300e0dfe9bad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0eae0e089734f02bb34300e0dfe9bad" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 34905 } } }
I20260812 03:08:39.188453 3967 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [sys.catalog]: This master's current role is: LEADER
W20260812 03:08:39.192279 3981 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:08:39.192356 3981 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:08:39.192440 3975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:08:39.193200 3975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:08:39.199544 3975 catalog_manager.cc:1347] Generated new cluster ID: 67800f693e3e4f4c81c3275c434978f1
I20260812 03:08:39.199640 3975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:08:39.214520 3975 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:08:39.215775 3975 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:08:39.223563 3975 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad: Generated new TSK 0
I20260812 03:08:39.224486 3975 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.250.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
I20260812 03:08:39.266558 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:0
--local_ip_for_outbound_sockets=127.26.250.129
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:34905
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:36383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:39.389462 3987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:39.389760 3987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:39.389845 3987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:39.394244 3987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:39.394318 3987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:39.394398 3987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:08:39.400183 3987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:36383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:34905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:39.401430 3987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:39.402635 3987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:39.413380 3996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:39.413355 3993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:39.413380 3994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:39.414505 3987 server_base.cc:1034] running on GCE node
I20260812 03:08:39.414927 3987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:39.415558 3987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:39.416770 3987 hybrid_clock.cc:648] HybridClock initialized: now 1786504119416735 us; error 58 us; skew 500 ppm
Aug 12 03:08:39 dist-test-slave-zpfg krb5kdc[3877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504119, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:39.419737 3987 init.cc:377] Logged in from keytab as kudu/127.26.250.129@KRBTEST.COM (short username kudu)
I20260812 03:08:39.420584 3987 webserver.cc:460] Webserver started at http://127.26.250.129:39323/ using document root <none> and password file <none>
I20260812 03:08:39.421113 3987 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:39.421170 3987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:39.421347 3987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:39.423481 3987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/data/instance:
uuid: "4a713e1b34b6415084580db9633a3d78"
format_stamp: "Formatted at 2026-08-12 03:08:39 on dist-test-slave-zpfg"
server_key: "ef417f4ff64454e3e07aa16665907829"
server_key_iv: "5bd0d02c23613992e04ec17045d46907"
server_key_version: "encryptionkey@0"
I20260812 03:08:39.423996 3987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance:
uuid: "4a713e1b34b6415084580db9633a3d78"
format_stamp: "Formatted at 2026-08-12 03:08:39 on dist-test-slave-zpfg"
server_key: "ef417f4ff64454e3e07aa16665907829"
server_key_iv: "5bd0d02c23613992e04ec17045d46907"
server_key_version: "encryptionkey@0"
I20260812 03:08:39.428079 3987 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260812 03:08:39.430812 4003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:39.432099 3987 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:08:39.432205 3987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "4a713e1b34b6415084580db9633a3d78"
format_stamp: "Formatted at 2026-08-12 03:08:39 on dist-test-slave-zpfg"
server_key: "ef417f4ff64454e3e07aa16665907829"
server_key_iv: "5bd0d02c23613992e04ec17045d46907"
server_key_version: "encryptionkey@0"
I20260812 03:08:39.432358 3987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:39.449133 3987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:39.452611 3987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:39.452827 3987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:39.453480 3987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:39.454418 3987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:39.454486 3987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:39.454559 3987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:39.454602 3987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:39.464172 3987 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:37855
I20260812 03:08:39.464193 4123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:37855 every 8 connection(s)
I20260812 03:08:39.465123 3987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:08:39.475373 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 3987
I20260812 03:08:39.475507 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance
Aug 12 03:08:39 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504119, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:39.475878 27626 external_mini_cluster.cc:1428] Setting key c56b5565dc6e7ec9ca508b4c4fba5203
I20260812 03:08:39.480232 3964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:39.467231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:49749 (local address 127.26.250.190:34905)
0812 03:08:39.467410 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:39.467415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:39.468202 (+ 787us) server_negotiation.cc:407] Connection header received
0812 03:08:39.469609 (+ 1407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:39.469612 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:39.469663 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:39.469743 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:39.471352 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:39.471900 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:39.473115 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:39.473355 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:39.473443 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:39.476367 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:39.476372 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:39.476377 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:39.476411 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:39.478016 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:39.478628 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:39.478633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:39.478635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:39.478724 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:39.479079 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:39.479082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:39.479084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:39.479275 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:39.479383 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:39.479913 (+ 530us) server_negotiation.cc:299] Negotiation successful
0812 03:08:39.480043 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:08:39.481161 4127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:39.467544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.129:49749)
0812 03:08:39.468036 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:39.468083 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:39.469398 (+ 1315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:39.469965 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:39.469972 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:39.470638 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:39.471193 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:39.471209 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:39.472025 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:39.472028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:39.472994 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:39.473004 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:39.473528 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:39.473531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:39.473728 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:39.474334 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:39.474414 (+ 80us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:39.476188 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:39.478137 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:39.478143 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:39.478161 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:39.478523 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:39.478820 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:39.478823 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:39.478825 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:39.478973 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:39.479387 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:39.479402 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:39.479660 (+ 258us) client_negotiation.cc:769] Sending connection context
0812 03:08:39.479851 (+ 191us) client_negotiation.cc:240] Negotiation successful
0812 03:08:39.480200 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":124,"threads_started":1}
I20260812 03:08:39.482584 4125 heartbeater.cc:347] Connected to a master server at 127.26.250.190:34905
I20260812 03:08:39.482857 4125 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:39.483608 4125 heartbeater.cc:511] Master 127.26.250.190:34905 requested a full tablet report, sending...
I20260812 03:08:39.485379 3923 ts_manager.cc:194] Registered new tserver with Master: 4a713e1b34b6415084580db9633a3d78 (127.26.250.129:37855)
I20260812 03:08:39.486660 3923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.129@KRBTEST.COM'} at 127.26.250.129:49749
WARNING: no policy specified for kudu/127.26.250.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
I20260812 03:08:39.510437 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:0
--local_ip_for_outbound_sockets=127.26.250.130
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:34905
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:36383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:39.631225 4131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:39.631548 4131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:39.631625 4131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:39.635982 4131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:39.636082 4131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:39.636318 4131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:08:39.642060 4131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:36383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:34905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:39.643317 4131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:39.644409 4131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:39.654838 4137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:39.654857 4138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:39.655071 4131 server_base.cc:1034] running on GCE node
W20260812 03:08:39.654856 4140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:39.655495 4131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:39.656204 4131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:39.657407 4131 hybrid_clock.cc:648] HybridClock initialized: now 1786504119657378 us; error 43 us; skew 500 ppm
Aug 12 03:08:39 dist-test-slave-zpfg krb5kdc[3877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504119, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:39.660527 4131 init.cc:377] Logged in from keytab as kudu/127.26.250.130@KRBTEST.COM (short username kudu)
I20260812 03:08:39.661478 4131 webserver.cc:460] Webserver started at http://127.26.250.130:35609/ using document root <none> and password file <none>
I20260812 03:08:39.662092 4131 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:39.662192 4131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:39.662510 4131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:39.664712 4131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/data/instance:
uuid: "5ce5ad0d75994402b9e37a3e1437f63d"
format_stamp: "Formatted at 2026-08-12 03:08:39 on dist-test-slave-zpfg"
server_key: "af868a80adca0349229b53dbbdb9c5d3"
server_key_iv: "3f729cfd77e6d8fe94dcca727db3951e"
server_key_version: "encryptionkey@0"
I20260812 03:08:39.665356 4131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance:
uuid: "5ce5ad0d75994402b9e37a3e1437f63d"
format_stamp: "Formatted at 2026-08-12 03:08:39 on dist-test-slave-zpfg"
server_key: "af868a80adca0349229b53dbbdb9c5d3"
server_key_iv: "3f729cfd77e6d8fe94dcca727db3951e"
server_key_version: "encryptionkey@0"
I20260812 03:08:39.669723 4131 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 03:08:39.672581 4147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:39.673997 4131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:08:39.674166 4131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "5ce5ad0d75994402b9e37a3e1437f63d"
format_stamp: "Formatted at 2026-08-12 03:08:39 on dist-test-slave-zpfg"
server_key: "af868a80adca0349229b53dbbdb9c5d3"
server_key_iv: "3f729cfd77e6d8fe94dcca727db3951e"
server_key_version: "encryptionkey@0"
I20260812 03:08:39.674301 4131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:39.692092 4131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:39.695767 4131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:39.696002 4131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:39.696677 4131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:39.697564 4131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:39.697639 4131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:39.697715 4131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:39.697769 4131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:39.707661 4131 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:37833
I20260812 03:08:39.707688 4264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:37833 every 8 connection(s)
I20260812 03:08:39.708669 4131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:08:39.718542 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 4131
I20260812 03:08:39.718698 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance
Aug 12 03:08:39 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504119, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:39.719010 27626 external_mini_cluster.cc:1428] Setting key 85aca0aa87e0296308b179f19793eff9
I20260812 03:08:39.723546 3964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:39.710567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:50935 (local address 127.26.250.190:34905)
0812 03:08:39.710740 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:39.710745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:39.711707 (+ 962us) server_negotiation.cc:407] Connection header received
0812 03:08:39.713098 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:39.713101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:39.713152 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:39.713231 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:39.714515 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:39.715047 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:39.716200 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:39.716561 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:39.716660 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:39.719618 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:39.719622 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:39.719624 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:39.719652 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:39.721241 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:39.721817 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:39.721823 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:39.721829 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:39.721909 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:39.722296 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:39.722299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:39.722301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:39.722563 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:39.722677 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:39.723286 (+ 609us) server_negotiation.cc:299] Negotiation successful
0812 03:08:39.723409 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 03:08:39.724888 4267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:39.711052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.130:50935)
0812 03:08:39.711547 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:39.711592 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:39.712892 (+ 1300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:39.713401 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:39.713408 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:39.713796 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:39.714372 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:39.714386 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:39.715164 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:39.715166 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:39.716052 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:39.716061 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:39.716717 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:39.716720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:39.716910 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:39.717645 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:39.717662 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:39.719426 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:39.721373 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:39.721379 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:39.721397 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:39.721695 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:39.722009 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:39.722012 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:39.722014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:39.722179 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:39.722680 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:39.722690 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:39.722973 (+ 283us) client_negotiation.cc:769] Sending connection context
0812 03:08:39.723764 (+ 791us) client_negotiation.cc:240] Negotiation successful
0812 03:08:39.724059 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":126,"threads_started":1}
I20260812 03:08:39.726182 4265 heartbeater.cc:347] Connected to a master server at 127.26.250.190:34905
I20260812 03:08:39.726460 4265 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:39.727005 4265 heartbeater.cc:511] Master 127.26.250.190:34905 requested a full tablet report, sending...
I20260812 03:08:39.728122 3923 ts_manager.cc:194] Registered new tserver with Master: 5ce5ad0d75994402b9e37a3e1437f63d (127.26.250.130:37833)
I20260812 03:08:39.728716 3923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.130@KRBTEST.COM'} at 127.26.250.130:50935
WARNING: no policy specified for kudu/127.26.250.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
I20260812 03:08:39.750084 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:0
--local_ip_for_outbound_sockets=127.26.250.131
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:34905
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:36383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:39.870268 4270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:39.870590 4270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:39.870636 4270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:39.874682 4270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:39.874751 4270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:39.874819 4270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:08:39.880566 4270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:36383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:34905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:39.881700 4270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:39.882830 4270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:39.894367 4278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:39.894402 4276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:39.894413 4275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:39.894719 4270 server_base.cc:1034] running on GCE node
I20260812 03:08:39.895188 4270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:39.895808 4270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:39.896984 4270 hybrid_clock.cc:648] HybridClock initialized: now 1786504119896958 us; error 37 us; skew 500 ppm
Aug 12 03:08:39 dist-test-slave-zpfg krb5kdc[3877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504119, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:39.900223 4270 init.cc:377] Logged in from keytab as kudu/127.26.250.131@KRBTEST.COM (short username kudu)
I20260812 03:08:39.901172 4270 webserver.cc:460] Webserver started at http://127.26.250.131:37761/ using document root <none> and password file <none>
I20260812 03:08:39.901729 4270 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:39.901789 4270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:39.901974 4270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:39.904196 4270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/data/instance:
uuid: "75a59d94315441dc91c01eb1fdff03a7"
format_stamp: "Formatted at 2026-08-12 03:08:39 on dist-test-slave-zpfg"
server_key: "31ca1bd8e5977a32fc2fb40bc2566809"
server_key_iv: "d3dd8ccc3070dcc24d76dc585af20c41"
server_key_version: "encryptionkey@0"
I20260812 03:08:39.904780 4270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance:
uuid: "75a59d94315441dc91c01eb1fdff03a7"
format_stamp: "Formatted at 2026-08-12 03:08:39 on dist-test-slave-zpfg"
server_key: "31ca1bd8e5977a32fc2fb40bc2566809"
server_key_iv: "d3dd8ccc3070dcc24d76dc585af20c41"
server_key_version: "encryptionkey@0"
I20260812 03:08:39.909065 4270 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260812 03:08:39.911887 4285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:39.913203 4270 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:08:39.913359 4270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "75a59d94315441dc91c01eb1fdff03a7"
format_stamp: "Formatted at 2026-08-12 03:08:39 on dist-test-slave-zpfg"
server_key: "31ca1bd8e5977a32fc2fb40bc2566809"
server_key_iv: "d3dd8ccc3070dcc24d76dc585af20c41"
server_key_version: "encryptionkey@0"
I20260812 03:08:39.913489 4270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:39.925400 4270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:39.929003 4270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:39.929237 4270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:39.929885 4270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:39.930897 4270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:39.930971 4270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:39.931046 4270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:39.931098 4270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:39.940896 4270 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:42635
I20260812 03:08:39.940980 4405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:42635 every 8 connection(s)
I20260812 03:08:39.941888 4270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:08:39.948273 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 4270
I20260812 03:08:39.948433 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance
I20260812 03:08:39.948731 27626 external_mini_cluster.cc:1428] Setting key 1be031f2cfbd5018d6059e21e87c4223
Aug 12 03:08:39 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504119, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:39.956748 3964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:39.943932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:52883 (local address 127.26.250.190:34905)
0812 03:08:39.944079 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:39.944083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:39.944992 (+ 909us) server_negotiation.cc:407] Connection header received
0812 03:08:39.946487 (+ 1495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:39.946490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:39.946544 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:39.946615 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:39.947970 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:39.948825 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:39.950285 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:39.950632 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:39.950743 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:39.953361 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:39.953365 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:39.953367 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:39.953394 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:39.954796 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:39.955291 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:39.955294 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:39.955295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:39.955407 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:39.955713 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:39.955715 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:39.955716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:39.955909 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:39.955991 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:39.956494 (+ 503us) server_negotiation.cc:299] Negotiation successful
0812 03:08:39.956613 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:08:39.957721 4408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:39.944338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.131:52883)
0812 03:08:39.944805 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:39.944858 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:39.946175 (+ 1317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:39.946792 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:39.946800 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:39.947188 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:39.947745 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:39.947760 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:39.948964 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:39.948967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:39.950138 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:39.950151 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:39.950786 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:39.950789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:39.951008 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:39.951672 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:39.951689 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:39.953190 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:39.954910 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:39.954915 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:39.954928 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:39.955197 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:39.955490 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:39.955492 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:39.955493 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:39.955628 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:39.956001 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:39.956006 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:39.956272 (+ 266us) client_negotiation.cc:769] Sending connection context
0812 03:08:39.956442 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 03:08:39.956717 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":129,"threads_started":1}
I20260812 03:08:39.959103 4406 heartbeater.cc:347] Connected to a master server at 127.26.250.190:34905
I20260812 03:08:39.959347 4406 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:39.959919 4406 heartbeater.cc:511] Master 127.26.250.190:34905 requested a full tablet report, sending...
I20260812 03:08:39.960987 3923 ts_manager.cc:194] Registered new tserver with Master: 75a59d94315441dc91c01eb1fdff03a7 (127.26.250.131:42635)
I20260812 03:08:39.961599 3923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.131@KRBTEST.COM'} at 127.26.250.131:52883
I20260812 03:08:39.963186 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.26.250.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.250.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.250.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.129.keytab.
Entry for principal oryx/127.26.250.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.129.keytab.
WARNING: no policy specified for oryx/127.26.250.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.250.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.250.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.130.keytab.
Entry for principal oryx/127.26.250.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.130.keytab.
WARNING: no policy specified for oryx/127.26.250.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.26.250.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.26.250.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.131.keytab.
Entry for principal oryx/127.26.250.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.131.keytab.
I20260812 03:08:40.053239 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 3987
I20260812 03:08:40.061992 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 4131
I20260812 03:08:40.069635 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 4270
I20260812 03:08:40.078529 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 3891
I20260812 03:08:40.087270 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:34905
--webserver_interface=127.26.250.190
--webserver_port=40381
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:36383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:34905
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:40.208088 4415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:40.208403 4415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:40.208456 4415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:40.212590 4415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:08:40.212667 4415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:40.212687 4415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:40.212704 4415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:08:40.212719 4415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:08:40.219035 4415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:36383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:34905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:34905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=40381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:40.220203 4415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:40.221222 4415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:40.230628 4423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:40.230655 4420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:40.230628 4421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:40.230945 4415 server_base.cc:1034] running on GCE node
I20260812 03:08:40.231686 4415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:40.232823 4415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:40.234040 4415 hybrid_clock.cc:648] HybridClock initialized: now 1786504120234006 us; error 50 us; skew 500 ppm
Aug 12 03:08:40 dist-test-slave-zpfg krb5kdc[3877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504120, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:40.237012 4415 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:08:40.237957 4415 webserver.cc:460] Webserver started at http://127.26.250.190:40381/ using document root <none> and password file <none>
I20260812 03:08:40.238595 4415 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:40.238677 4415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:40.242834 4415 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 03:08:40.245251 4430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:40.246717 4415 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:08:40.246878 4415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "a0eae0e089734f02bb34300e0dfe9bad"
format_stamp: "Formatted at 2026-08-12 03:08:39 on dist-test-slave-zpfg"
server_key: "ebd02d7d9149f5d6a80fdf43ae9aa10b"
server_key_iv: "03d65d0f18160005b357f810b6dc7e60"
server_key_version: "encryptionkey@0"
I20260812 03:08:40.247500 4415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:40.265407 4415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:40.268833 4415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:40.269063 4415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:40.277482 4415 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:34905
I20260812 03:08:40.277505 4485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:34905 every 8 connection(s)
I20260812 03:08:40.278815 4415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:08:40.283567 4486 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:08:40.285197 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 4415
I20260812 03:08:40.285550 4486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad: Bootstrap starting.
I20260812 03:08:40.285759 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:37855
--local_ip_for_outbound_sockets=127.26.250.129
--tserver_master_addrs=127.26.250.190:34905
--webserver_port=39323
--webserver_interface=127.26.250.129
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:36383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.26.250.129
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:08:40.304507 4486 log.cc:824] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:40.311800 4486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:08:40.312300 4486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad: Bootstrap complete.
I20260812 03:08:40.316882 4486 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0eae0e089734f02bb34300e0dfe9bad" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 34905 } }
I20260812 03:08:40.317319 4486 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0eae0e089734f02bb34300e0dfe9bad, State: Initialized, Role: FOLLOWER
I20260812 03:08:40.317878 4486 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0eae0e089734f02bb34300e0dfe9bad" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 34905 } }
I20260812 03:08:40.318118 4486 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:08:40.318186 4486 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:08:40.318307 4486 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:08:40.320430 4486 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0eae0e089734f02bb34300e0dfe9bad" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 34905 } }
I20260812 03:08:40.320787 4486 leader_election.cc:302] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0eae0e089734f02bb34300e0dfe9bad; no voters:
I20260812 03:08:40.321151 4486 leader_election.cc:288] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 03:08:40.321247 4491 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:08:40.321480 4491 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [term 2 LEADER]: Becoming Leader. State: Replica: a0eae0e089734f02bb34300e0dfe9bad, State: Running, Role: LEADER
I20260812 03:08:40.321863 4491 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0eae0e089734f02bb34300e0dfe9bad" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 34905 } }
I20260812 03:08:40.322507 4486 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:08:40.322870 4491 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0eae0e089734f02bb34300e0dfe9bad. Latest consensus state: current_term: 2 leader_uuid: "a0eae0e089734f02bb34300e0dfe9bad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0eae0e089734f02bb34300e0dfe9bad" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 34905 } } }
I20260812 03:08:40.323679 4491 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:40.323551 4493 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a0eae0e089734f02bb34300e0dfe9bad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0eae0e089734f02bb34300e0dfe9bad" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 34905 } } }
I20260812 03:08:40.324463 4503 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:08:40.324478 4493 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:40.327903 4503 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:08:40.328562 4503 catalog_manager.cc:1259] Loaded cluster ID: 67800f693e3e4f4c81c3275c434978f1
I20260812 03:08:40.328629 4503 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:08:40.331034 4503 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:08:40.331719 4503 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a0eae0e089734f02bb34300e0dfe9bad: Loaded TSK: 0
I20260812 03:08:40.332933 4503 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:08:40.409917 4488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:40.410238 4488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:40.410284 4488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:40.414495 4488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:40.414561 4488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:40.414656 4488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:08:40.420261 4488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:36383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.129.keytab
--principal=oryx/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:37855
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=39323
--tserver_master_addrs=127.26.250.190:34905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:40.421449 4488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:40.422626 4488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:40.433272 4517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:40.433290 4518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:40.433455 4488 server_base.cc:1034] running on GCE node
W20260812 03:08:40.433609 4520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:40.434021 4488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:40.434823 4488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:40.436029 4488 hybrid_clock.cc:648] HybridClock initialized: now 1786504120436013 us; error 33 us; skew 500 ppm
Aug 12 03:08:40 dist-test-slave-zpfg krb5kdc[3877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504120, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:40.439200 4488 init.cc:377] Logged in from keytab as oryx/127.26.250.129@KRBTEST.COM (short username oryx)
I20260812 03:08:40.440128 4488 webserver.cc:460] Webserver started at http://127.26.250.129:39323/ using document root <none> and password file <none>
I20260812 03:08:40.440739 4488 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:40.440855 4488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:40.445317 4488 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:08:40.447753 4527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:40.449070 4488 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:08:40.449218 4488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "4a713e1b34b6415084580db9633a3d78"
format_stamp: "Formatted at 2026-08-12 03:08:39 on dist-test-slave-zpfg"
server_key: "ef417f4ff64454e3e07aa16665907829"
server_key_iv: "5bd0d02c23613992e04ec17045d46907"
server_key_version: "encryptionkey@0"
I20260812 03:08:40.449823 4488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:40.461113 4488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:40.465080 4488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:40.465334 4488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:40.466029 4488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:40.467064 4488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:40.467147 4488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:40.467240 4488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:40.467303 4488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:40.477659 4488 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:37855
I20260812 03:08:40.477685 4645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:37855 every 8 connection(s)
I20260812 03:08:40.478826 4488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:08:40.483932 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 4488
I20260812 03:08:40.484560 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:37833
--local_ip_for_outbound_sockets=127.26.250.130
--tserver_master_addrs=127.26.250.190:34905
--webserver_port=35609
--webserver_interface=127.26.250.130
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:36383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.26.250.130
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:08:40 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:40 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:40.494187 4649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:40.481160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.129:58397)
0812 03:08:40.481632 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:40.481687 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:40.483115 (+ 1428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:40.484256 (+ 1141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:40.484263 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:40.484790 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:40.485305 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:40.485318 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.486976 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.486979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:40.488307 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:40.488323 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.489129 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.489137 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:40.489420 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:40.490151 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:40.490179 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:40.493156 (+ 2977us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":150,"threads_started":1}
W20260812 03:08:40.494179 4648 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:40.481012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:58397 (local address 127.26.250.190:34905)
0812 03:08:40.481499 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:40.481515 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:40.481810 (+ 295us) server_negotiation.cc:407] Connection header received
0812 03:08:40.483516 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:40.483527 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:40.483834 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:40.484179 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:40.485497 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.486803 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.488456 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.488935 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.489063 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:40.493410 (+ 4347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.129:58397: BlockingRecv error: recv got EOF from 127.26.250.129:58397 (error 108)
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":129,"threads_started":1}
W20260812 03:08:40.494865 4646 heartbeater.cc:644] Failed to heartbeat to 127.26.250.190:34905 (0 consecutive failures): Not authorized: Failed to ping master at 127.26.250.190:34905: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Aug 12 03:08:40 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:40 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:40.500695 4649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:40.495372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.129:44933)
0812 03:08:40.495497 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:40.495511 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:40.495616 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:40.495857 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:40.495860 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:40.496031 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:40.496122 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:40.496139 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.496887 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.496893 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:40.497444 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:40.497452 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.497940 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.497944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:40.498046 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:40.498627 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:40.498638 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:40.500541 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 03:08:40.500792 4648 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:40.495494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:44933 (local address 127.26.250.190:34905)
0812 03:08:40.495630 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:40.495636 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:40.495661 (+ 25us) server_negotiation.cc:407] Connection header received
0812 03:08:40.495704 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:40.495706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:40.495756 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:40.495849 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:40.496249 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.496769 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.497572 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.497795 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.497955 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:40.500669 (+ 2714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.129:44933: BlockingRecv error: recv got EOF from 127.26.250.129:44933 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 03:08:40 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:40 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:40.506790 4649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:40.501276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.129:42213)
0812 03:08:40.501443 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:40.501456 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:40.501524 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:40.501751 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:40.501754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:40.501924 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:40.502019 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:40.502025 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.502808 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.502814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:40.503737 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:40.503747 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.504158 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.504161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:40.504257 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:40.504720 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:40.504736 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:40.506617 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102}
W20260812 03:08:40.506839 4648 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:40.501356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:42213 (local address 127.26.250.190:34905)
0812 03:08:40.501475 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:40.501478 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:40.501490 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:08:40.501603 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:40.501605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:40.501639 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:40.501703 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:40.502122 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.502662 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.503860 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.504058 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.504128 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:40.506742 (+ 2614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.129:42213: BlockingRecv error: recv got EOF from 127.26.250.129:42213 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260812 03:08:40.507025 4646 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:08:40.610173 4650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:40.610690 4650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:40.610775 4650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:40.615178 4650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:40.615289 4650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:40.615392 4650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:08:40.621172 4650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:36383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.130.keytab
--principal=oryx/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:37833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=35609
--tserver_master_addrs=127.26.250.190:34905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:40.622546 4650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:40.623767 4650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:40.634291 4658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:40.634321 4656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:40.634497 4650 server_base.cc:1034] running on GCE node
W20260812 03:08:40.634398 4655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:40.635026 4650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:40.635695 4650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:40.636894 4650 hybrid_clock.cc:648] HybridClock initialized: now 1786504120636880 us; error 28 us; skew 500 ppm
Aug 12 03:08:40 dist-test-slave-zpfg krb5kdc[3877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504120, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:40.639892 4650 init.cc:377] Logged in from keytab as oryx/127.26.250.130@KRBTEST.COM (short username oryx)
I20260812 03:08:40.640815 4650 webserver.cc:460] Webserver started at http://127.26.250.130:35609/ using document root <none> and password file <none>
I20260812 03:08:40.641403 4650 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:40.641505 4650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:40.645510 4650 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:08:40.647763 4665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:40.649133 4650 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:08:40.649266 4650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "5ce5ad0d75994402b9e37a3e1437f63d"
format_stamp: "Formatted at 2026-08-12 03:08:39 on dist-test-slave-zpfg"
server_key: "af868a80adca0349229b53dbbdb9c5d3"
server_key_iv: "3f729cfd77e6d8fe94dcca727db3951e"
server_key_version: "encryptionkey@0"
I20260812 03:08:40.649857 4650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:40.665853 4650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:40.669504 4650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:40.669724 4650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:40.670550 4650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:40.671454 4650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:40.671504 4650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:40.671572 4650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:40.671608 4650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:40.681070 4650 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:37833
I20260812 03:08:40.681140 4784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:37833 every 8 connection(s)
I20260812 03:08:40.682168 4650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:08:40.683454 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 4650
I20260812 03:08:40.683995 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:42635
--local_ip_for_outbound_sockets=127.26.250.131
--tserver_master_addrs=127.26.250.190:34905
--webserver_port=37761
--webserver_interface=127.26.250.131
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:36383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.26.250.131
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:08:40 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:40 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:08:40.696779 4648 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:40.684522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:52185 (local address 127.26.250.190:34905)
0812 03:08:40.684718 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:40.684726 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:40.685642 (+ 916us) server_negotiation.cc:407] Connection header received
0812 03:08:40.687553 (+ 1911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:40.687556 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:40.687610 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:40.687688 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:40.689583 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.690117 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.692106 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.692325 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.692413 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:40.696642 (+ 4229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.130:52185: BlockingRecv error: recv got EOF from 127.26.250.130:52185 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 03:08:40.697563 4787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:40.684860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.130:52185)
0812 03:08:40.685431 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:40.685498 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:40.687301 (+ 1803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:40.687915 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:40.687927 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:40.688585 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:40.689402 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:40.689429 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.690271 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.690277 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:40.691970 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:40.691986 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.692510 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.692514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:40.692753 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:40.693454 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:40.693481 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:40.696486 (+ 3005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":130,"threads_started":1}
W20260812 03:08:40.698221 4785 heartbeater.cc:644] Failed to heartbeat to 127.26.250.190:34905 (0 consecutive failures): Not authorized: Failed to ping master at 127.26.250.190:34905: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Aug 12 03:08:40 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:40 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:40.704249 4787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:40.698722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.130:56711)
0812 03:08:40.698977 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:40.698992 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:40.699099 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:40.699385 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:40.699388 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:40.699611 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:40.699716 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:40.699722 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.700492 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.700497 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:40.701062 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:40.701070 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.701514 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.701518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:40.701612 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:40.702134 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:40.702147 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:40.704143 (+ 1996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189}
W20260812 03:08:40.704471 4648 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:40.698855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:56711 (local address 127.26.250.190:34905)
0812 03:08:40.698966 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:40.698970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:40.699039 (+ 69us) server_negotiation.cc:407] Connection header received
0812 03:08:40.699230 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:40.699232 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:40.699274 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:40.699367 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:40.699853 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.700379 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.701180 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.701402 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.701509 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:40.704344 (+ 2835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.130:56711: BlockingRecv error: recv got EOF from 127.26.250.130:56711 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 12 03:08:40 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:40 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:40.710788 4787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:40.705023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.130:54593)
0812 03:08:40.705220 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:40.705232 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:40.705295 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:40.705529 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:40.705531 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:40.705656 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:40.705729 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:40.705734 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.706802 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.706805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:40.707341 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:40.707350 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.707976 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.707978 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:40.708041 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:40.708603 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:40.708615 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:40.710678 (+ 2063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123}
W20260812 03:08:40.711087 4785 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:08:40.710963 4648 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:40.705136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:54593 (local address 127.26.250.190:34905)
0812 03:08:40.705279 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:40.705284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:40.705300 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:08:40.705372 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:40.705375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:40.705419 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:40.705503 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:40.705832 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.706653 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.707482 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.707838 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.707948 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:40.710845 (+ 2897us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.130:54593: BlockingRecv error: recv got EOF from 127.26.250.130:54593 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260812 03:08:40.812631 4788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:40.812949 4788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:40.812996 4788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:40.817085 4788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:40.817158 4788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:40.817220 4788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:08:40.822999 4788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:36383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/krb5kdc/oryx_127.26.250.131.keytab
--principal=oryx/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:42635
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=37761
--tserver_master_addrs=127.26.250.190:34905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:40.824328 4788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:40.825559 4788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:40.836522 4796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:40.836522 4793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:40.836648 4794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:40.836663 4788 server_base.cc:1034] running on GCE node
I20260812 03:08:40.837309 4788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:40.837992 4788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:40.839255 4788 hybrid_clock.cc:648] HybridClock initialized: now 1786504120839230 us; error 34 us; skew 500 ppm
Aug 12 03:08:40 dist-test-slave-zpfg krb5kdc[3877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504120, etypes {rep=17 tkt=17 ses=17}, oryx/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:40.842478 4788 init.cc:377] Logged in from keytab as oryx/127.26.250.131@KRBTEST.COM (short username oryx)
I20260812 03:08:40.843439 4788 webserver.cc:460] Webserver started at http://127.26.250.131:37761/ using document root <none> and password file <none>
I20260812 03:08:40.844024 4788 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:40.844127 4788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:40.848259 4788 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:08:40.850692 4804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:40.852520 4788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:08:40.852658 4788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "75a59d94315441dc91c01eb1fdff03a7"
format_stamp: "Formatted at 2026-08-12 03:08:39 on dist-test-slave-zpfg"
server_key: "31ca1bd8e5977a32fc2fb40bc2566809"
server_key_iv: "d3dd8ccc3070dcc24d76dc585af20c41"
server_key_version: "encryptionkey@0"
I20260812 03:08:40.853266 4788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:40.884546 4788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:40.888353 4788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:40.888587 4788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:40.889274 4788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:40.890159 4788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:40.890249 4788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:40.890336 4788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:40.890396 4788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260812 03:08:40.900408 4788 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:42635
I20260812 03:08:40.900441 4920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:42635 every 8 connection(s)
I20260812 03:08:40.901510 4788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:08:40.903334 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 4788
Aug 12 03:08:40 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:40 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
W20260812 03:08:40.914150 4648 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:40.903705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:43335 (local address 127.26.250.190:34905)
0812 03:08:40.903861 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:40.903868 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:40.905048 (+ 1180us) server_negotiation.cc:407] Connection header received
0812 03:08:40.906877 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:40.906880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:40.906941 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:40.907023 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:40.908309 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.908986 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.910166 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.910459 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.910572 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:40.914023 (+ 3451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.131:43335: BlockingRecv error: recv got EOF from 127.26.250.131:43335 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:08:40.914605 4925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:40.904639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36520 (local address 127.26.250.190:34905)
0812 03:08:40.904890 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:40.904895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:40.904909 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:08:40.904955 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:40.904958 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:40.905020 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:40.905117 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:40.906230 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.906920 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.907845 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.908111 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.908235 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:40.909844 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:40.909850 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:40.909866 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:40.909897 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:40.911983 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:40.912485 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:40.912491 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:40.912497 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:40.912659 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:40.912981 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:40.912985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:40.912987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:40.913429 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:40.913641 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:40.913937 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 03:08:40.914481 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":81,"threads_started":1}
I20260812 03:08:40.914970 4924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:40.904167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.131:43335)
0812 03:08:40.904896 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:40.904942 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:40.906647 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:40.907233 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:40.907241 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:40.907614 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:40.908150 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:40.908165 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.909120 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.909124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:40.910034 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:40.910043 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.910655 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.910661 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:40.910849 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:40.911491 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:40.911508 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:40.913917 (+ 2409us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":189,"threads_started":1}
W20260812 03:08:40.915575 4921 heartbeater.cc:644] Failed to heartbeat to 127.26.250.190:34905 (0 consecutive failures): Not authorized: Failed to ping master at 127.26.250.190:34905: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Aug 12 03:08:40 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:40 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:40.921607 4924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:40.916112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.131:51335)
0812 03:08:40.916274 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:40.916290 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:40.916382 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:40.916663 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:40.916669 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:40.916843 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:40.916972 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:40.916979 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.917848 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.917850 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:40.918503 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:40.918513 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.918988 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.918991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:40.919058 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:40.919563 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:40.919575 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:40.921496 (+ 1921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":79}
W20260812 03:08:40.922055 4925 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:40.916220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:51335 (local address 127.26.250.190:34905)
0812 03:08:40.916331 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:40.916334 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:40.916347 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:08:40.916487 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:40.916490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:40.916546 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:40.916625 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:40.917109 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.917731 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.918638 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.918888 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.918973 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:40.921730 (+ 2757us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.131:51335: BlockingRecv error: recv got EOF from 127.26.250.131:51335 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 03:08:40 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:40 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:40.927697 4924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:40.922221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.131:56867)
0812 03:08:40.922515 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:40.922530 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:40.922619 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:40.922879 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:40.922882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:40.923094 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:40.923188 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:40.923193 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.923990 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.923994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:40.924655 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:40.924663 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.925105 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.925109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:40.925189 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:40.925687 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:40.925699 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:40.927541 (+ 1842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219}
W20260812 03:08:40.927776 4648 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:40.922300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:56867 (local address 127.26.250.190:34905)
0812 03:08:40.922526 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:40.922531 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:40.922575 (+ 44us) server_negotiation.cc:407] Connection header received
0812 03:08:40.922726 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:40.922728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:40.922775 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:40.922865 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:40.923316 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.923878 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.924768 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:40.924995 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:40.925104 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:40.927660 (+ 2556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.131:56867: BlockingRecv error: recv got EOF from 127.26.250.131:56867 (error 108)
Metrics: {"server-negotiator.queue_time_us":151}
W20260812 03:08:40.928032 4921 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 03:08:41 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:41 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:41.514286 4926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:41.507941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.129:51019)
0812 03:08:41.508420 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:41.508444 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:41.508592 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:41.508988 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:41.508991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:41.509253 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:41.509398 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:41.509407 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:41.510191 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:41.510193 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:41.510841 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:41.510852 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:41.511289 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:41.511291 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:41.511387 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:41.511928 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:41.511941 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:41.514148 (+ 2207us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":117,"threads_started":1}
W20260812 03:08:41.514539 4927 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:41.508121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:51019 (local address 127.26.250.190:34905)
0812 03:08:41.508437 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:41.508442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:41.508548 (+ 106us) server_negotiation.cc:407] Connection header received
0812 03:08:41.508762 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:41.508768 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:41.508840 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:41.508950 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:41.509521 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:41.510092 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:41.510951 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:41.511184 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:41.511269 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:41.514325 (+ 3056us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.129:51019: BlockingRecv error: recv got EOF from 127.26.250.129:51019 (error 108)
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":101,"threads_started":1}
Aug 12 03:08:41 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:41 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:41.717700 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:41.711907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.130:42487)
0812 03:08:41.712241 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:41.712265 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:41.712463 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:41.712751 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:41.712754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:41.712978 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:41.713080 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:41.713085 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:41.713899 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:41.713904 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:41.714515 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:41.714523 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:41.715004 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:41.715007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:41.715092 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:41.715634 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:41.715645 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:41.717568 (+ 1923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":95,"threads_started":1}
W20260812 03:08:41.717885 4927 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:41.712023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:42487 (local address 127.26.250.190:34905)
0812 03:08:41.712203 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:41.712207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:41.712392 (+ 185us) server_negotiation.cc:407] Connection header received
0812 03:08:41.712571 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:41.712576 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:41.712631 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:41.712722 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:41.713207 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:41.713749 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:41.714640 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:41.714879 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:41.714998 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:41.717712 (+ 2714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.130:42487: BlockingRecv error: recv got EOF from 127.26.250.130:42487 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 12 03:08:41 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:41 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:41.936091 4929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:41.928957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.131:59555)
0812 03:08:41.929547 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:41.929565 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:41.929707 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:41.930067 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:41.930073 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:41.930279 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:41.930504 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:41.930513 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:41.931539 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:41.931543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:41.932292 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:41.932303 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:41.932834 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:41.932837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:41.932904 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:41.933800 (+ 896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:41.933814 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:41.935941 (+ 2127us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":92,"threads_started":1}
W20260812 03:08:41.936278 4927 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:41.929114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:59555 (local address 127.26.250.190:34905)
0812 03:08:41.929281 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:41.929288 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:41.929797 (+ 509us) server_negotiation.cc:407] Connection header received
0812 03:08:41.929856 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:41.929859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:41.929922 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:41.930003 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:41.930663 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:41.931404 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:41.932473 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:41.932710 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:41.932804 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:41.936097 (+ 3293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.131:59555: BlockingRecv error: recv got EOF from 127.26.250.131:59555 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 12 03:08:42 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:42 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:42.522063 4930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:42.515305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.129:51765)
0812 03:08:42.515616 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:42.515689 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:42.515800 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:42.516262 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:42.516265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:42.516468 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:42.516585 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:42.516591 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:42.517552 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:42.517555 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:42.518207 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:42.518218 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:42.518821 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:42.518824 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:42.518896 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:42.519499 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:42.519512 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:42.521932 (+ 2420us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":77,"threads_started":1}
W20260812 03:08:42.522476 4931 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:42.515463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:51765 (local address 127.26.250.190:34905)
0812 03:08:42.515771 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:42.515775 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:42.515790 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:08:42.515932 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:42.515935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:42.515988 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:42.516075 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:42.516734 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:42.517422 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:42.518372 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:42.518682 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:42.518780 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:42.522260 (+ 3480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.129:51765: BlockingRecv error: recv got EOF from 127.26.250.129:51765 (error 108)
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":81,"threads_started":1}
Aug 12 03:08:42 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:42 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:42.724431 4932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:42.718647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.130:35595)
0812 03:08:42.718939 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:42.718953 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:42.719064 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:42.719312 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:42.719314 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:42.719507 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:42.719613 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:42.719619 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:42.720352 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:42.720354 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:42.720923 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:42.720931 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:42.721522 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:42.721525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:42.721596 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:42.722135 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:42.722147 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:42.724300 (+ 2153us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":77,"threads_started":1}
W20260812 03:08:42.724642 4931 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:42.718745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:35595 (local address 127.26.250.190:34905)
0812 03:08:42.718935 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:42.718942 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:42.719031 (+ 89us) server_negotiation.cc:407] Connection header received
0812 03:08:42.719144 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:42.719146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:42.719194 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:42.719265 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:42.719717 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:42.720248 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:42.721058 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:42.721380 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:42.721478 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:42.724502 (+ 3024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.130:35595: BlockingRecv error: recv got EOF from 127.26.250.130:35595 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 12 03:08:42 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:42 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:42.942955 4933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:42.936981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.131:55257)
0812 03:08:42.937326 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:42.937344 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:42.937497 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:42.937773 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:42.937776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:42.938018 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:42.938117 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:42.938123 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:42.938987 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:42.938993 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:42.939577 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:42.939585 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:42.940068 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:42.940071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:42.940161 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:42.940751 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:42.940763 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:42.942804 (+ 2041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":134,"threads_started":1}
W20260812 03:08:42.943128 4931 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:42.937160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:55257 (local address 127.26.250.190:34905)
0812 03:08:42.937321 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:42.937326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:42.937431 (+ 105us) server_negotiation.cc:407] Connection header received
0812 03:08:42.937601 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:42.937604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:42.937657 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:42.937748 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:42.938242 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:42.938871 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:42.939702 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:42.939936 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:42.940047 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:42.942963 (+ 2916us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.131:55257: BlockingRecv error: recv got EOF from 127.26.250.131:55257 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 12 03:08:43 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:43 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:43.529116 4934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:43.523043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.129:42903)
0812 03:08:43.523402 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:43.523422 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:43.523522 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:43.523779 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:43.523781 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:43.523964 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:43.524060 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:43.524084 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:43.524899 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:43.524902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:43.525457 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:43.525465 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:43.525992 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:43.525999 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:43.526085 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:43.526733 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:43.526745 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:43.528962 (+ 2217us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":112,"threads_started":1}
W20260812 03:08:43.529280 4935 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:43.523104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:42903 (local address 127.26.250.190:34905)
0812 03:08:43.523513 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:43.523517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:43.523534 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:08:43.523595 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:43.523598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:43.523653 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:43.523739 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:43.524170 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:43.524780 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:43.525566 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:43.525789 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:43.525969 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:43.529109 (+ 3140us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.129:42903: BlockingRecv error: recv got EOF from 127.26.250.129:42903 (error 108)
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":79,"threads_started":1}
Aug 12 03:08:43 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:43 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:43.731484 4937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:43.725437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.130:56111)
0812 03:08:43.725776 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:43.725795 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:43.725964 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:43.726291 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:43.726293 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:43.726615 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:43.726729 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:43.726735 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:43.727555 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:43.727558 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:43.728157 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:43.728166 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:43.728634 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:43.728637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:43.728703 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:43.729320 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:43.729335 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:43.731327 (+ 1992us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":99,"threads_started":1}
W20260812 03:08:43.731699 4935 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:43.725467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:56111 (local address 127.26.250.190:34905)
0812 03:08:43.725637 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:43.725642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:43.725891 (+ 249us) server_negotiation.cc:407] Connection header received
0812 03:08:43.726088 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:43.726091 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:43.726140 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:43.726211 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:43.726872 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:43.727438 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:43.728277 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:43.728515 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:43.728602 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:43.731548 (+ 2946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.130:56111: BlockingRecv error: recv got EOF from 127.26.250.130:56111 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 12 03:08:43 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:43 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:43.949522 4938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:43.943970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.131:49513)
0812 03:08:43.944264 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:43.944282 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:43.944482 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:43.944741 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:43.944743 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:43.944931 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:43.945031 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:43.945036 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:43.945855 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:43.945858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:43.946460 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:43.946468 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:43.946904 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:43.946906 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:43.946969 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:43.947489 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:43.947501 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:43.949398 (+ 1897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":113,"threads_started":1}
W20260812 03:08:43.949680 4935 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:43.944015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:49513 (local address 127.26.250.190:34905)
0812 03:08:43.944181 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:43.944187 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:43.944395 (+ 208us) server_negotiation.cc:407] Connection header received
0812 03:08:43.944569 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:43.944571 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:43.944624 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:43.944711 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:43.945165 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:43.945749 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:43.946566 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:43.946787 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:43.946882 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:43.949549 (+ 2667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.131:49513: BlockingRecv error: recv got EOF from 127.26.250.131:49513 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 12 03:08:44 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:44 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:44.540566 4939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:44.529990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.129:51249)
0812 03:08:44.530323 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:44.530366 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:44.530469 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:44.530736 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:44.530739 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:44.530957 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:44.531061 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:44.531066 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:44.531880 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:44.531883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:44.532550 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:44.532560 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:44.533027 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:44.533029 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:44.533093 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:44.538227 (+ 5134us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:44.538241 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:44.540440 (+ 2199us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":99,"threads_started":1}
W20260812 03:08:44.540758 4940 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:44.530050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:51249 (local address 127.26.250.190:34905)
0812 03:08:44.530456 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:44.530460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:44.530475 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:08:44.530539 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:44.530541 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:44.530583 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:44.530682 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:44.531183 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:44.531767 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:44.532693 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:44.532922 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:44.533005 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:44.540601 (+ 7596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.129:51249: BlockingRecv error: recv got EOF from 127.26.250.129:51249 (error 108)
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":202,"threads_started":1}
Aug 12 03:08:44 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:44 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:44.737769 4941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:44.732249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.130:49309)
0812 03:08:44.732521 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:44.732535 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:44.732638 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:44.732871 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:44.732873 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:44.733059 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:44.733164 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:44.733170 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:44.733989 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:44.733992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:44.734619 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:44.734629 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:44.735044 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:44.735046 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:44.735108 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:44.735601 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:44.735612 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:44.737646 (+ 2034us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":84,"threads_started":1}
W20260812 03:08:44.737967 4940 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:44.732352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:49309 (local address 127.26.250.190:34905)
0812 03:08:44.732538 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:44.732542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:44.732607 (+ 65us) server_negotiation.cc:407] Connection header received
0812 03:08:44.732715 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:44.732717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:44.732762 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:44.732839 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:44.733295 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:44.733890 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:44.734717 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:44.734942 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:44.735024 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:44.737793 (+ 2769us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.130:49309: BlockingRecv error: recv got EOF from 127.26.250.130:49309 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 12 03:08:44 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:44 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:44.956698 4942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:44.950558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.131:55087)
0812 03:08:44.950952 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:44.950965 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:44.951115 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:44.951395 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:44.951398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:44.951621 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:44.951724 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:44.951731 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:44.952606 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:44.952611 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:44.953174 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:44.953182 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:44.953709 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:44.953713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:44.953798 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:44.954489 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:44.954502 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:44.956492 (+ 1990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":118,"threads_started":1}
W20260812 03:08:44.956797 4940 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:44.950634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:55087 (local address 127.26.250.190:34905)
0812 03:08:44.950895 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:44.950901 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:44.951046 (+ 145us) server_negotiation.cc:407] Connection header received
0812 03:08:44.951233 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:44.951236 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:44.951296 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:44.951368 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:44.951882 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:44.952482 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:44.953309 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:44.953586 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:44.953703 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:44.956660 (+ 2957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.131:55087: BlockingRecv error: recv got EOF from 127.26.250.131:55087 (error 108)
Metrics: {"server-negotiator.queue_time_us":160}
Aug 12 03:08:45 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:45 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:45.547559 4943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:45.541498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.129:57515)
0812 03:08:45.541835 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:45.541849 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:45.541949 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:45.542242 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:45.542245 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:45.542553 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:45.542666 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:45.542673 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:45.543456 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:45.543462 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:45.544062 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:45.544071 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:45.544541 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:45.544545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:45.544632 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:45.545221 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:45.545233 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:45.547428 (+ 2195us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":99,"threads_started":1}
W20260812 03:08:45.547745 4944 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:45.541563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:57515 (local address 127.26.250.190:34905)
0812 03:08:45.541852 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:45.541858 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:45.541909 (+ 51us) server_negotiation.cc:407] Connection header received
0812 03:08:45.542065 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:45.542068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:45.542116 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:45.542220 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:45.542787 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:45.543343 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:45.544185 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:45.544427 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:45.544536 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:45.547571 (+ 3035us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.129:57515: BlockingRecv error: recv got EOF from 127.26.250.129:57515 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":68,"threads_started":1}
Aug 12 03:08:45 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:45 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:45.744295 4945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:45.738725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.130:57821)
0812 03:08:45.739055 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:45.739069 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:45.739168 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:45.739434 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:45.739436 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:45.739631 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:45.739732 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:45.739737 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:45.740485 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:45.740487 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:45.741067 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:45.741074 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:45.741509 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:45.741512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:45.741578 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:45.742078 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:45.742090 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:45.744113 (+ 2023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":99,"threads_started":1}
W20260812 03:08:45.744380 4944 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:45.738941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:57821 (local address 127.26.250.190:34905)
0812 03:08:45.739084 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:45.739088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:45.739122 (+ 34us) server_negotiation.cc:407] Connection header received
0812 03:08:45.739249 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:45.739251 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:45.739295 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:45.739393 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:45.739831 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:45.740379 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:45.741165 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:45.741401 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:45.741481 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:45.744252 (+ 2771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.130:57821: BlockingRecv error: recv got EOF from 127.26.250.130:57821 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 03:08:45 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:45 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:45.963647 4946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:45.957752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.131:54219)
0812 03:08:45.958067 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:45.958080 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:45.958208 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:45.958580 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:45.958584 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:45.958786 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:45.958895 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:45.958902 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:45.959654 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:45.959657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:45.960252 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:45.960261 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:45.960728 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:45.960731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:45.960793 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:45.961281 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:45.961292 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:45.963418 (+ 2126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":108,"threads_started":1}
W20260812 03:08:45.963743 4944 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:45.957778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:54219 (local address 127.26.250.190:34905)
0812 03:08:45.957953 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:45.957959 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:45.958158 (+ 199us) server_negotiation.cc:407] Connection header received
0812 03:08:45.958313 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:45.958316 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:45.958405 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:45.958503 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:45.959013 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:45.959550 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:45.960411 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:45.960630 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:45.960707 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:45.963594 (+ 2887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.131:54219: BlockingRecv error: recv got EOF from 127.26.250.131:54219 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 03:08:46 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:46 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:46.554468 4947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:46.548385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.129:57977)
0812 03:08:46.548690 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:46.548703 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:46.548810 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:46.549045 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:46.549047 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:46.549241 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:46.549396 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:46.549402 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:46.550202 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:46.550205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:46.550875 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:46.550885 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:46.551400 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:46.551404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:46.551490 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:46.552115 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:46.552127 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:46.554280 (+ 2153us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":96,"threads_started":1}
W20260812 03:08:46.554766 4948 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:46.548445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:57977 (local address 127.26.250.190:34905)
0812 03:08:46.548710 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:46.548714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:46.548825 (+ 111us) server_negotiation.cc:407] Connection header received
0812 03:08:46.548876 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:46.548878 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:46.548926 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:46.549012 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:46.549528 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:46.550102 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:46.551024 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:46.551277 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:46.551393 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:46.554516 (+ 3123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.129:57977: BlockingRecv error: recv got EOF from 127.26.250.129:57977 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":63,"threads_started":1}
Aug 12 03:08:46 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:46 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:46.750963 4949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:46.745101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.130:42659)
0812 03:08:46.745436 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:46.745451 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:46.745566 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:46.745797 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:46.745799 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:46.745999 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:46.746102 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:46.746107 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:46.746858 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:46.746861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:46.747433 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:46.747440 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:46.747866 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:46.747868 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:46.747934 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:46.748543 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:46.748554 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:46.750770 (+ 2216us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":115,"threads_started":1}
W20260812 03:08:46.751130 4948 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:46.745196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:42659 (local address 127.26.250.190:34905)
0812 03:08:46.745359 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:46.745365 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:46.745523 (+ 158us) server_negotiation.cc:407] Connection header received
0812 03:08:46.745645 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:46.745647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:46.745691 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:46.745759 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:46.746203 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:46.746752 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:46.747533 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:46.747753 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:46.747853 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:46.750959 (+ 3106us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.130:42659: BlockingRecv error: recv got EOF from 127.26.250.130:42659 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 12 03:08:46 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:46 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:46.970139 4950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:46.964524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.131:55167)
0812 03:08:46.964851 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:46.964864 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:46.965003 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:46.965273 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:46.965276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:46.965494 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:46.965592 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:46.965597 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:46.966470 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:46.966473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:46.967044 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:46.967053 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:46.967548 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:46.967551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:46.967614 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:46.968119 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:46.968131 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:46.970012 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":91,"threads_started":1}
W20260812 03:08:46.970326 4948 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:46.964622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:55167 (local address 127.26.250.190:34905)
0812 03:08:46.964784 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:46.964791 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:46.964941 (+ 150us) server_negotiation.cc:407] Connection header received
0812 03:08:46.965103 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:46.965106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:46.965160 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:46.965246 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:46.965717 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:46.966304 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:46.967182 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:46.967428 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:46.967530 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:46.970158 (+ 2628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.131:55167: BlockingRecv error: recv got EOF from 127.26.250.131:55167 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 12 03:08:47 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:47 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:47.561237 4951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:47.555324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.129:43513)
0812 03:08:47.555710 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:47.555724 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:47.555840 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:47.556093 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:47.556096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:47.556298 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:47.556413 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:47.556420 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:47.557284 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:47.557287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:47.557976 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:47.557987 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:47.558520 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:47.558522 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:47.558589 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:47.559235 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:47.559248 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:47.561110 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":112,"threads_started":1}
W20260812 03:08:47.561437 4952 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:47.555434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:43513 (local address 127.26.250.190:34905)
0812 03:08:47.555712 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:47.555716 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:47.555825 (+ 109us) server_negotiation.cc:407] Connection header received
0812 03:08:47.555928 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:47.555930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:47.555977 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:47.556058 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:47.556518 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:47.557171 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:47.558120 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:47.558415 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:47.558500 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:47.561261 (+ 2761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.129:43513: BlockingRecv error: recv got EOF from 127.26.250.129:43513 (error 108)
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":97,"threads_started":1}
Aug 12 03:08:47 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:47 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:47.758504 4953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:47.752023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.130:45807)
0812 03:08:47.752356 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:47.752370 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:47.752478 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:47.752703 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:47.752705 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:47.752898 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:47.752997 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:47.753002 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:47.753850 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:47.753853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:47.754596 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:47.754607 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:47.755220 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:47.755223 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:47.755312 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:47.756090 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:47.756106 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:47.758292 (+ 2186us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":93,"threads_started":1}
W20260812 03:08:47.758666 4952 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:47.752049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:45807 (local address 127.26.250.190:34905)
0812 03:08:47.752228 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:47.752233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:47.752429 (+ 196us) server_negotiation.cc:407] Connection header received
0812 03:08:47.752563 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:47.752566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:47.752616 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:47.752693 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:47.753098 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:47.753704 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:47.754741 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:47.755097 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:47.755209 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:47.758515 (+ 3306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.130:45807: BlockingRecv error: recv got EOF from 127.26.250.130:45807 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 12 03:08:47 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:47 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:47.977831 4954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:47.971233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.131:34915)
0812 03:08:47.971588 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:47.971607 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:47.971727 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:47.972057 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:47.972060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:47.972351 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:47.972468 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:47.972480 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:47.973323 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:47.973326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:47.973960 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:47.973971 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:47.974594 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:47.974597 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:47.974703 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:47.975391 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:47.975427 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:47.977685 (+ 2258us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":75,"threads_started":1}
W20260812 03:08:47.978179 4952 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:47.971452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:34915 (local address 127.26.250.190:34905)
0812 03:08:47.971612 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:47.971615 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:47.971688 (+ 73us) server_negotiation.cc:407] Connection header received
0812 03:08:47.971852 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:47.971854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:47.971905 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:47.971982 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:47.972601 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:47.973161 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:47.974129 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:47.974469 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:47.974589 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:47.977875 (+ 3286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.131:34915: BlockingRecv error: recv got EOF from 127.26.250.131:34915 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 12 03:08:48 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:48 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:48.568678 4955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:48.562132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.129:40717)
0812 03:08:48.562457 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:48.562471 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:48.562570 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:48.563093 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:48.563095 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:48.563397 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:48.563508 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:48.563515 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:48.564465 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:48.564468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:48.565065 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:48.565075 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:48.565615 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:48.565618 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:48.565695 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:48.566270 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:48.566283 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:48.568497 (+ 2214us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":105,"threads_started":1}
W20260812 03:08:48.568920 4956 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:48.562437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:40717 (local address 127.26.250.190:34905)
0812 03:08:48.562698 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:48.562703 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:48.562718 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:08:48.562757 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:48.562759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:48.562801 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:48.562901 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:48.563691 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:48.564340 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:48.565262 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:48.565505 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:48.565592 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:48.568717 (+ 3125us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.129:40717: BlockingRecv error: recv got EOF from 127.26.250.129:40717 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":104,"threads_started":1}
Aug 12 03:08:48 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:48 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:48.765594 4957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:48.759386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.130:44107)
0812 03:08:48.759743 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:48.759762 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:48.759921 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:48.760217 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:48.760220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:48.760476 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:48.760583 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:48.760590 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:48.761511 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:48.761515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:48.762086 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:48.762094 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:48.762629 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:48.762633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:48.762721 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:48.763426 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:48.763439 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:48.765469 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":100,"threads_started":1}
W20260812 03:08:48.765740 4956 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:48.759537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:44107 (local address 127.26.250.190:34905)
0812 03:08:48.759724 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:48.759729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:48.759892 (+ 163us) server_negotiation.cc:407] Connection header received
0812 03:08:48.760039 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:48.760042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:48.760095 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:48.760185 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:48.760713 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:48.761395 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:48.762208 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:48.762502 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:48.762619 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:48.765617 (+ 2998us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.130:44107: BlockingRecv error: recv got EOF from 127.26.250.130:44107 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 12 03:08:48 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:48 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:48.984921 4958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:48.978822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.131:54757)
0812 03:08:48.979130 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:48.979146 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:48.979272 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:48.979574 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:48.979577 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:48.979778 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:48.979886 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:48.979892 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:48.980719 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:48.980722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:48.981346 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:48.981414 (+ 68us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:48.981953 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:48.981957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:48.982047 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:48.982698 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:48.982716 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:48.984798 (+ 2082us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":122,"threads_started":1}
W20260812 03:08:48.985046 4956 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:48.978926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:54757 (local address 127.26.250.190:34905)
0812 03:08:48.979063 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:48.979066 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:48.979207 (+ 141us) server_negotiation.cc:407] Connection header received
0812 03:08:48.979372 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:48.979374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:48.979422 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:48.979489 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:48.980019 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:48.980601 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:48.981538 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:48.981821 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:48.981935 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:48.984926 (+ 2991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.131:54757: BlockingRecv error: recv got EOF from 127.26.250.131:54757 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 12 03:08:49 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:49 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:49.585099 4959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:49.569778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.129:47215)
0812 03:08:49.570087 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:49.570100 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:49.570191 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:49.570509 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:49.570511 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:49.570695 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:49.570793 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:49.570799 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:49.571744 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:49.571747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:49.572308 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:49.572316 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:49.572825 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:49.572827 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:49.572900 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:49.582778 (+ 9878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:49.582794 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:49.584967 (+ 2173us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":109,"threads_started":1}
W20260812 03:08:49.585379 4960 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:49.569846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:47215 (local address 127.26.250.190:34905)
0812 03:08:49.570126 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:49.570130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:49.570258 (+ 128us) server_negotiation.cc:407] Connection header received
0812 03:08:49.570306 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:49.570309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:49.570396 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:49.570474 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:49.570923 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:49.571623 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:49.572444 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:49.572711 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:49.572819 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:49.585194 (+ 12375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.129:47215: BlockingRecv error: recv got EOF from 127.26.250.129:47215 (error 108)
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":74,"threads_started":1}
Aug 12 03:08:49 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:49 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:49.772910 4961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:49.766674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.130:48617)
0812 03:08:49.766995 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:49.767017 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:49.767126 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:49.767417 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:49.767420 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:49.767614 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:49.767734 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:49.767741 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:49.768571 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:49.768574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:49.769301 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:49.769313 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:49.769863 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:49.769867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:49.769965 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:49.770641 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:49.770654 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:49.772735 (+ 2081us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":124,"threads_started":1}
W20260812 03:08:49.773000 4960 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:49.766810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:48617 (local address 127.26.250.190:34905)
0812 03:08:49.766995 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:49.767000 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:49.767078 (+ 78us) server_negotiation.cc:407] Connection header received
0812 03:08:49.767199 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:49.767202 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:49.767246 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:49.767377 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:49.767858 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:49.768422 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:49.769459 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:49.769706 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:49.769849 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:49.772901 (+ 3052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.130:48617: BlockingRecv error: recv got EOF from 127.26.250.130:48617 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 12 03:08:49 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:49 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:49.991909 4962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:49.985978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.131:33157)
0812 03:08:49.986431 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:49.986445 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:49.986579 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:49.986858 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:49.986861 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:49.987089 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:49.987183 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:49.987188 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:49.987955 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:49.987961 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:49.988623 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:49.988632 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:49.989090 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:49.989094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:49.989180 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:49.989741 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:49.989753 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:49.991727 (+ 1974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":125,"threads_started":1}
W20260812 03:08:49.991986 4960 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:49.986126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:33157 (local address 127.26.250.190:34905)
0812 03:08:49.986332 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:49.986365 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:49.986529 (+ 164us) server_negotiation.cc:407] Connection header received
0812 03:08:49.986685 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:49.986689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:49.986741 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:49.986865 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:49.987297 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:49.987843 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:49.988738 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:49.988975 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:49.989085 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:49.991868 (+ 2783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.131:33157: BlockingRecv error: recv got EOF from 127.26.250.131:33157 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 12 03:08:50 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:50 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:50.591993 4963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:50.585951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.129:55153)
0812 03:08:50.586280 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:50.586305 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:50.586453 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:50.586713 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:50.586715 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:50.586936 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:50.587044 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:50.587050 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:50.587875 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:50.587878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:50.588513 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:50.588522 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:50.588968 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:50.588970 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:50.589040 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:50.589586 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:50.589600 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:50.591871 (+ 2271us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":82,"threads_started":1}
W20260812 03:08:50.592173 4964 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:50.586109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:55153 (local address 127.26.250.190:34905)
0812 03:08:50.586425 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:50.586428 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:50.586444 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:08:50.586544 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:50.586546 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:50.586589 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:50.586666 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:50.587151 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:50.587741 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:50.588635 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:50.588862 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:50.588940 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:50.592028 (+ 3088us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.129:55153: BlockingRecv error: recv got EOF from 127.26.250.129:55153 (error 108)
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":88,"threads_started":1}
Aug 12 03:08:50 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:50 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:50.782440 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:50.774072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.130:44437)
0812 03:08:50.774549 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:50.774567 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:50.774702 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:50.775159 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:50.775163 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:50.775437 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:50.775608 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:50.775618 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:50.776920 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:50.776927 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:50.778033 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:50.778047 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:50.778823 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:50.778834 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:50.778958 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:50.779745 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:50.779763 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:50.782237 (+ 2474us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":109,"threads_started":1}
W20260812 03:08:50.782668 4964 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:50.774335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:44437 (local address 127.26.250.190:34905)
0812 03:08:50.774545 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:50.774552 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:50.774665 (+ 113us) server_negotiation.cc:407] Connection header received
0812 03:08:50.774894 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:50.774898 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:50.774970 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:50.775096 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:50.775828 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:50.776770 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:50.778213 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:50.778682 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:50.778799 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:50.782556 (+ 3757us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.130:44437: BlockingRecv error: recv got EOF from 127.26.250.130:44437 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 12 03:08:50 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:50 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:50.999114 4966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:50.992943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.131:34575)
0812 03:08:50.993266 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:50.993286 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:50.993410 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:50.993704 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:50.993707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:50.993946 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:50.994054 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:50.994060 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:50.994909 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:50.994911 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:50.995515 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:50.995524 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:50.995985 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:50.995987 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:50.996055 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:50.996698 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:50.996711 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:50.998983 (+ 2272us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":116,"threads_started":1}
W20260812 03:08:50.999397 4964 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:50.993120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:34575 (local address 127.26.250.190:34905)
0812 03:08:50.993269 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:50.993272 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:50.993361 (+ 89us) server_negotiation.cc:407] Connection header received
0812 03:08:50.993501 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:50.993504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:50.993552 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:50.993629 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:50.994169 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:50.994771 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:50.995633 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:50.995873 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:50.995960 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:50.999249 (+ 3289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.131:34575: BlockingRecv error: recv got EOF from 127.26.250.131:34575 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 12 03:08:51 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:51 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:51.598861 4967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:51.592876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.129:40299)
0812 03:08:51.593216 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:51.593231 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:51.593353 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:51.593674 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:51.593676 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:51.593922 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:51.594026 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:51.594032 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:51.594936 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:51.594939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:51.595512 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:51.595520 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:51.595988 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:51.595991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:51.596057 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:51.596622 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:51.596634 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:51.598706 (+ 2072us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":95,"threads_started":1}
W20260812 03:08:51.599092 4968 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:51.592993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:40299 (local address 127.26.250.190:34905)
0812 03:08:51.593265 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:51.593268 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:51.593283 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:08:51.593497 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:51.593500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:51.593551 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:51.593633 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:51.594142 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:51.594796 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:51.595627 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:51.595883 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:51.595966 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:51.598928 (+ 2962us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.129:40299: BlockingRecv error: recv got EOF from 127.26.250.129:40299 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":65,"threads_started":1}
Aug 12 03:08:51 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:51 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:51.790273 4969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:51.783466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.130:46145)
0812 03:08:51.783987 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:51.784005 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:51.784128 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:51.784450 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:51.784454 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:51.784723 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:51.784870 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:51.784880 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:51.785770 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:51.785773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:51.786516 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:51.786527 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:51.787119 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:51.787276 (+ 157us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:51.787408 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:51.788015 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:51.788029 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:51.790110 (+ 2081us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":103,"threads_started":1}
W20260812 03:08:51.790447 4968 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:51.783528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:46145 (local address 127.26.250.190:34905)
0812 03:08:51.783769 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:51.783774 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:51.784104 (+ 330us) server_negotiation.cc:407] Connection header received
0812 03:08:51.784231 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:51.784235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:51.784300 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:51.784378 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:51.785010 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:51.785639 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:51.786639 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:51.786915 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:51.787026 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:51.790280 (+ 3254us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.130:46145: BlockingRecv error: recv got EOF from 127.26.250.130:46145 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 12 03:08:52 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:52 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:52.006872 4970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:52.000385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.131:57691)
0812 03:08:52.000887 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:52.000907 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:52.001065 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:52.001399 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:52.001402 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:52.001670 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:52.001822 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:52.001830 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:52.002684 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:52.002687 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:52.003343 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:52.003353 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:52.003794 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:52.003803 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:52.003871 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:52.004545 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:52.004559 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:52.006684 (+ 2125us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":95,"threads_started":1}
W20260812 03:08:52.006978 4968 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:52.000589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:57691 (local address 127.26.250.190:34905)
0812 03:08:52.000743 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:52.000750 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:52.001020 (+ 270us) server_negotiation.cc:407] Connection header received
0812 03:08:52.001175 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:52.001178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:52.001244 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:52.001368 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:52.001964 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:52.002577 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:52.003456 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:52.003680 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:52.003762 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:52.006848 (+ 3086us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.131:57691: BlockingRecv error: recv got EOF from 127.26.250.131:57691 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 12 03:08:52 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:52 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:52.606304 4971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:52.599947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.129:43197)
0812 03:08:52.600342 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:52.600357 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:52.600462 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:52.600786 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:52.600788 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:52.600989 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:52.601095 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:52.601101 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:52.601961 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:52.601964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:52.602758 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:52.602768 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:52.603320 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:52.603327 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:52.603416 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:52.604067 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:52.604086 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:52.606075 (+ 1989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":298,"spinlock_wait_cycles":12032,"thread_start_us":230,"threads_started":1}
W20260812 03:08:52.606395 4972 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:52.600138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:43197 (local address 127.26.250.190:34905)
0812 03:08:52.600421 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:52.600429 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:52.600452 (+ 23us) server_negotiation.cc:407] Connection header received
0812 03:08:52.600557 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:52.600559 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:52.600608 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:52.600750 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:52.601238 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:52.601837 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:52.602877 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:52.603188 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:52.603309 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:52.606222 (+ 2913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.129:43197: BlockingRecv error: recv got EOF from 127.26.250.129:43197 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":70,"threads_started":1}
Aug 12 03:08:52 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:52 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:52.797293 4973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:52.791340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.130:33325)
0812 03:08:52.791636 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:52.791650 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:52.791762 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:52.792018 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:52.792020 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:52.792211 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:52.792314 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:52.792319 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:52.793199 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:52.793202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:52.793771 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:52.793780 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:52.794272 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:52.794274 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:52.794398 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:52.794947 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:52.794958 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:52.797112 (+ 2154us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":84,"threads_started":1}
W20260812 03:08:52.797525 4972 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:52.791453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:33325 (local address 127.26.250.190:34905)
0812 03:08:52.791650 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:52.791654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:52.791718 (+ 64us) server_negotiation.cc:407] Connection header received
0812 03:08:52.791855 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:52.791857 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:52.791905 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:52.791980 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:52.792511 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:52.793081 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:52.793909 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:52.794161 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:52.794250 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:52.797248 (+ 2998us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.130:33325: BlockingRecv error: recv got EOF from 127.26.250.130:33325 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 12 03:08:53 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:53 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:53.014119 4974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:53.007892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.131:37603)
0812 03:08:53.008188 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:53.008204 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:53.008313 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:53.008564 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:53.008566 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:53.008806 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:53.008910 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:53.008916 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:53.009817 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:53.009820 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:53.010573 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:53.010586 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:53.011104 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:53.011107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:53.011180 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:53.011716 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:53.011729 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:53.013907 (+ 2178us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":99,"threads_started":1}
W20260812 03:08:53.014189 4972 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:53.008010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:37603 (local address 127.26.250.190:34905)
0812 03:08:53.008174 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:53.008178 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:53.008279 (+ 101us) server_negotiation.cc:407] Connection header received
0812 03:08:53.008403 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:53.008405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:53.008455 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:53.008521 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:53.009038 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:53.009647 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:53.010709 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:53.010962 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:53.011054 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:53.014065 (+ 3011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.131:37603: BlockingRecv error: recv got EOF from 127.26.250.131:37603 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 12 03:08:53 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:53 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:53.614009 4975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:53.607211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.129:50465)
0812 03:08:53.607572 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:53.607588 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:53.607699 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:53.608021 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:53.608023 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:53.608232 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:53.608349 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:53.608356 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:53.609188 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:53.609190 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:53.609868 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:53.609878 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:53.610573 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:53.610576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:53.610660 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:53.611398 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:53.611412 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:53.613813 (+ 2401us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":136,"threads_started":1}
W20260812 03:08:53.614109 4976 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:53.607311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:50465 (local address 127.26.250.190:34905)
0812 03:08:53.607635 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:53.607639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:53.607656 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:08:53.607792 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:53.607794 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:53.607882 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:53.607972 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:53.608475 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:53.609081 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:53.610042 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:53.610423 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:53.610522 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:53.613962 (+ 3440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.129:50465: BlockingRecv error: recv got EOF from 127.26.250.129:50465 (error 108)
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":101,"threads_started":1}
Aug 12 03:08:53 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:53 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:53.804930 4977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:53.798216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.130:60677)
0812 03:08:53.798580 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:53.798599 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:53.798746 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:53.799085 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:53.799088 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:53.799329 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:53.799436 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:53.799443 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:53.800257 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:53.800261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:53.800918 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:53.800929 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:53.801450 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:53.801454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:53.801553 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:53.802121 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:53.802134 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:53.804664 (+ 2530us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":134,"threads_started":1}
W20260812 03:08:53.805001 4976 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:53.798445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:60677 (local address 127.26.250.190:34905)
0812 03:08:53.798598 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:53.798603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:53.798676 (+ 73us) server_negotiation.cc:407] Connection header received
0812 03:08:53.798893 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:53.798896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:53.798951 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:53.799048 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:53.799574 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:53.800130 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:53.801061 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:53.801315 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:53.801425 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:53.804865 (+ 3440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.130:60677: BlockingRecv error: recv got EOF from 127.26.250.130:60677 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 03:08:54 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:54 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:54.021114 4978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:54.014983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.131:54239)
0812 03:08:54.015331 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:54.015345 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:54.015472 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:54.015766 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:54.015769 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:54.015994 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:54.016092 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:54.016097 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:54.016882 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:54.016887 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:54.017461 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:54.017469 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:54.017936 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:54.017940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:54.018041 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:54.018769 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:54.018781 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:54.020923 (+ 2142us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":94,"threads_started":1}
W20260812 03:08:54.021186 4976 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:54.015051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:54239 (local address 127.26.250.190:34905)
0812 03:08:54.015206 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:54.015212 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:54.015418 (+ 206us) server_negotiation.cc:407] Connection header received
0812 03:08:54.015578 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:54.015584 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:54.015644 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:54.015733 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:54.016221 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:54.016761 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:54.017581 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:54.017811 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:54.017915 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:54.021068 (+ 3153us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.131:54239: BlockingRecv error: recv got EOF from 127.26.250.131:54239 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 12 03:08:54 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:54 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:54.626150 4979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:54.615049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.129:53023)
0812 03:08:54.615352 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:54.615370 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:54.615491 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:54.615855 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:54.615857 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:54.616083 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:54.616224 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:54.616232 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:54.617386 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:54.617390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:54.618177 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:54.618188 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:54.618671 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:54.618674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:54.618776 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:54.623900 (+ 5124us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:54.623915 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:54.625997 (+ 2082us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":110,"threads_started":1}
W20260812 03:08:54.626341 4980 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:54.615205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:53023 (local address 127.26.250.190:34905)
0812 03:08:54.615604 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:54.615608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:54.615623 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:08:54.615666 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:54.615668 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:54.615711 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:54.615811 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:54.616439 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:54.617245 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:54.618296 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:54.618555 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:54.618637 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:54.626183 (+ 7546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.129:53023: BlockingRecv error: recv got EOF from 127.26.250.129:53023 (error 108)
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":173,"threads_started":1}
Aug 12 03:08:54 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:54 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:54.812721 4981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:54.806112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.130:53757)
0812 03:08:54.806559 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:54.806574 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:54.806687 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:54.806920 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:54.806922 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:54.807112 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:54.807216 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:54.807221 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:54.808317 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:54.808321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:54.809042 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:54.809053 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:54.809654 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:54.809662 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:54.809765 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:54.810497 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:54.810509 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:54.812581 (+ 2072us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":142,"threads_started":1}
W20260812 03:08:54.812951 4980 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:54.806209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:53757 (local address 127.26.250.190:34905)
0812 03:08:54.806448 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:54.806454 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:54.806659 (+ 205us) server_negotiation.cc:407] Connection header received
0812 03:08:54.806760 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:54.806762 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:54.806810 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:54.806885 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:54.807356 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:54.808070 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:54.809144 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:54.809487 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:54.809595 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:54.812750 (+ 3155us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.130:53757: BlockingRecv error: recv got EOF from 127.26.250.130:53757 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 12 03:08:55 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:55 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.131@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:55.028715 4982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:55.022047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.131:43415)
0812 03:08:55.022492 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:55.022510 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:55.022674 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:55.023002 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:55.023005 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:55.023248 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:55.023402 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:55.023413 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:55.024274 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:55.024280 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:55.024922 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:55.024931 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:55.025460 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:55.025464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:55.025561 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:55.026090 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:55.026103 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:55.028520 (+ 2417us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":95,"threads_started":1}
W20260812 03:08:55.028756 4980 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:55.022296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:43415 (local address 127.26.250.190:34905)
0812 03:08:55.022492 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:55.022499 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:55.022600 (+ 101us) server_negotiation.cc:407] Connection header received
0812 03:08:55.022783 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:55.022787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:55.022847 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:55.022946 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:55.023525 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:55.024161 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:55.025061 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:55.025323 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:55.025454 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:55.028646 (+ 3192us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.131:43415: BlockingRecv error: recv got EOF from 127.26.250.131:43415 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 12 03:08:55 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:55 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.129@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:55.633920 4983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:55.627258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.129:33115)
0812 03:08:55.627560 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:55.627575 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:55.627686 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:55.628001 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:55.628004 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:55.628250 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:55.628362 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:55.628369 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:55.629230 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:55.629233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:55.630004 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:55.630017 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:55.630647 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:55.630650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:55.630780 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:55.631498 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:55.631510 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:55.633767 (+ 2257us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":94,"threads_started":1}
W20260812 03:08:55.634100 4984 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:55.627369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:33115 (local address 127.26.250.190:34905)
0812 03:08:55.627630 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:55.627634 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:55.627651 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:08:55.627801 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:55.627815 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:55.627862 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:55.627943 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:55.628493 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:55.629067 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:55.630159 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:55.630485 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:55.630634 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:55.633953 (+ 3319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.129:33115: BlockingRecv error: recv got EOF from 127.26.250.129:33115 (error 108)
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":65,"threads_started":1}
Aug 12 03:08:55 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
Aug 12 03:08:55 dist-test-slave-zpfg krb5kdc[3877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.26.250.130@KRBTEST.COM for oryx/127.26.250.190@KRBTEST.COM, Server not found in Kerberos database
I20260812 03:08:55.820135 4985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:55.813753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:34905 (local address 127.26.250.130:53341)
0812 03:08:55.814038 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:55.814053 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:55.814154 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:55.814535 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:55.814538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:55.814821 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:55.814934 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:55.814941 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:55.815810 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:55.815815 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:55.816459 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:55.816468 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:55.817032 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:55.817036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:55.817143 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:55.817688 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:55.817701 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:55.819949 (+ 2248us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.26.250.190:34905: Server oryx/127.26.250.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":104,"threads_started":1}
W20260812 03:08:55.820232 4984 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:55.813830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:53341 (local address 127.26.250.190:34905)
0812 03:08:55.814000 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:55.814004 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:55.814151 (+ 147us) server_negotiation.cc:407] Connection header received
0812 03:08:55.814242 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:55.814244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:55.814330 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:55.814500 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:55.815082 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:55.815674 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:55.816597 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:55.816892 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:55.817011 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:55.820104 (+ 3093us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.26.250.130:53341: BlockingRecv error: recv got EOF from 127.26.250.130:53341 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 03:08:55.906080 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 4488
I20260812 03:08:55.915159 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 4650
I20260812 03:08:55.924981 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 4788
I20260812 03:08:55.933561 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 4415
2026-08-12T03:08:55Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18228 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T03:08:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:08:55Z Disabled control of system clock
I20260812 03:08:55.969337 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:36311
--webserver_interface=127.26.250.190
--webserver_port=0
--builtin_ntp_servers=127.26.250.148:34753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:36311
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 03:08:56.090121 4989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:56.090461 4989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:56.090499 4989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:56.094674 4989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:08:56.094748 4989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:56.094762 4989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:56.094779 4989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:08:56.094831 4989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:08:56.099897 4989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:34753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:36311
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:36311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:56.101111 4989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:56.102630 4989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:56.111771 4996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:56.111764 4995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:56.111850 4998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:56.111883 4989 server_base.cc:1034] running on GCE node
I20260812 03:08:56.112838 4989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:56.114053 4989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:56.115281 4989 hybrid_clock.cc:648] HybridClock initialized: now 1786504136115249 us; error 45 us; skew 500 ppm
I20260812 03:08:56.117525 4989 webserver.cc:460] Webserver started at http://127.26.250.190:35385/ using document root <none> and password file <none>
I20260812 03:08:56.118225 4989 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:56.118337 4989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:56.118640 4989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:56.120997 4989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "80e0bbff433a4b3a8e63f30d4ef0b477"
format_stamp: "Formatted at 2026-08-12 03:08:56 on dist-test-slave-zpfg"
server_key: "d3ed0959f82cc584d87196fa971b9c07"
server_key_iv: "540f30a1584043090d15b1109583967a"
server_key_version: "encryptionkey@0"
I20260812 03:08:56.121649 4989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "80e0bbff433a4b3a8e63f30d4ef0b477"
format_stamp: "Formatted at 2026-08-12 03:08:56 on dist-test-slave-zpfg"
server_key: "d3ed0959f82cc584d87196fa971b9c07"
server_key_iv: "540f30a1584043090d15b1109583967a"
server_key_version: "encryptionkey@0"
I20260812 03:08:56.126030 4989 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.007s
I20260812 03:08:56.129077 5004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:56.130838 4989 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:08:56.131021 4989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "80e0bbff433a4b3a8e63f30d4ef0b477"
format_stamp: "Formatted at 2026-08-12 03:08:56 on dist-test-slave-zpfg"
server_key: "d3ed0959f82cc584d87196fa971b9c07"
server_key_iv: "540f30a1584043090d15b1109583967a"
server_key_version: "encryptionkey@0"
I20260812 03:08:56.131284 4989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:56.153453 4989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:56.154410 4989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:56.154655 4989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:56.163465 4989 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:36311
I20260812 03:08:56.163501 5060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:36311 every 8 connection(s)
I20260812 03:08:56.164582 4989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:08:56.166683 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 4989
I20260812 03:08:56.166965 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:08:56.167284 27626 external_mini_cluster.cc:1428] Setting key f9c72373d206efaef25bbcd0bd31b62d
I20260812 03:08:56.168414 5061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:56.175347 5061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80e0bbff433a4b3a8e63f30d4ef0b477: Bootstrap starting.
I20260812 03:08:56.177296 5064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:56.168884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59360 (local address 127.26.250.190:36311)
0812 03:08:56.169545 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:56.169561 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:56.169598 (+ 37us) server_negotiation.cc:407] Connection header received
0812 03:08:56.170286 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:56.170313 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:56.170788 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:56.171082 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:56.171451 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:56.172441 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:56.173668 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:56.174165 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:56.174290 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:56.175276 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:56.175281 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:56.175294 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:08:56.175818 (+ 524us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:56.175880 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:08:56.175902 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:56.175982 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:56.176170 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 03:08:56.176522 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":151,"threads_started":1}
I20260812 03:08:56.177922 5061 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80e0bbff433a4b3a8e63f30d4ef0b477: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:56.178838 5061 log.cc:824] T 00000000000000000000000000000000 P 80e0bbff433a4b3a8e63f30d4ef0b477: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:56.181032 5061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80e0bbff433a4b3a8e63f30d4ef0b477: No bootstrap required, opened a new log
I20260812 03:08:56.184966 5061 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80e0bbff433a4b3a8e63f30d4ef0b477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80e0bbff433a4b3a8e63f30d4ef0b477" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 36311 } }
I20260812 03:08:56.185235 5061 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80e0bbff433a4b3a8e63f30d4ef0b477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:56.185319 5061 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80e0bbff433a4b3a8e63f30d4ef0b477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80e0bbff433a4b3a8e63f30d4ef0b477, State: Initialized, Role: FOLLOWER
I20260812 03:08:56.185804 5061 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80e0bbff433a4b3a8e63f30d4ef0b477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80e0bbff433a4b3a8e63f30d4ef0b477" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 36311 } }
I20260812 03:08:56.185959 5061 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80e0bbff433a4b3a8e63f30d4ef0b477 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:08:56.186048 5061 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80e0bbff433a4b3a8e63f30d4ef0b477 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:08:56.186152 5061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80e0bbff433a4b3a8e63f30d4ef0b477 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:56.187183 5061 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80e0bbff433a4b3a8e63f30d4ef0b477 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80e0bbff433a4b3a8e63f30d4ef0b477" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 36311 } }
I20260812 03:08:56.187548 5061 leader_election.cc:302] T 00000000000000000000000000000000 P 80e0bbff433a4b3a8e63f30d4ef0b477 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80e0bbff433a4b3a8e63f30d4ef0b477; no voters:
I20260812 03:08:56.187883 5061 leader_election.cc:288] T 00000000000000000000000000000000 P 80e0bbff433a4b3a8e63f30d4ef0b477 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:08:56.188060 5066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80e0bbff433a4b3a8e63f30d4ef0b477 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:56.188495 5066 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80e0bbff433a4b3a8e63f30d4ef0b477 [term 1 LEADER]: Becoming Leader. State: Replica: 80e0bbff433a4b3a8e63f30d4ef0b477, State: Running, Role: LEADER
I20260812 03:08:56.188841 5066 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80e0bbff433a4b3a8e63f30d4ef0b477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80e0bbff433a4b3a8e63f30d4ef0b477" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 36311 } }
I20260812 03:08:56.189221 5061 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80e0bbff433a4b3a8e63f30d4ef0b477 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:08:56.191082 5068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80e0bbff433a4b3a8e63f30d4ef0b477 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80e0bbff433a4b3a8e63f30d4ef0b477. Latest consensus state: current_term: 1 leader_uuid: "80e0bbff433a4b3a8e63f30d4ef0b477" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80e0bbff433a4b3a8e63f30d4ef0b477" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 36311 } } }
I20260812 03:08:56.191147 5067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80e0bbff433a4b3a8e63f30d4ef0b477 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80e0bbff433a4b3a8e63f30d4ef0b477" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80e0bbff433a4b3a8e63f30d4ef0b477" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 36311 } } }
I20260812 03:08:56.191231 5068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80e0bbff433a4b3a8e63f30d4ef0b477 [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:56.191264 5067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80e0bbff433a4b3a8e63f30d4ef0b477 [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:56.191610 5078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:08:56.194972 5078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:08:56.201009 5078 catalog_manager.cc:1347] Generated new cluster ID: 1c384ac2e8674e34a0dbc80142e4adea
I20260812 03:08:56.201105 5078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:08:56.219062 5078 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:08:56.220494 5078 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:08:56.230862 5078 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80e0bbff433a4b3a8e63f30d4ef0b477: Generated new TSK 0
I20260812 03:08:56.231842 5078 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:08:56.234891 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:0
--local_ip_for_outbound_sockets=127.26.250.129
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:36311
--builtin_ntp_servers=127.26.250.148:34753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 03:08:56.358604 5085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:56.358912 5085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:56.358984 5085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:56.363183 5085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:56.363248 5085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:56.363351 5085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:08:56.368240 5085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:34753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:36311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:56.369463 5085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:56.371419 5085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:56.384243 5092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:56.384285 5091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:56.384495 5085 server_base.cc:1034] running on GCE node
W20260812 03:08:56.384228 5094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:56.385070 5085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:56.385737 5085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:56.386922 5085 hybrid_clock.cc:648] HybridClock initialized: now 1786504136386903 us; error 34 us; skew 500 ppm
I20260812 03:08:56.388983 5085 webserver.cc:460] Webserver started at http://127.26.250.129:35319/ using document root <none> and password file <none>
I20260812 03:08:56.389626 5085 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:56.389740 5085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:56.390026 5085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:56.392446 5085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/data/instance:
uuid: "ec8bf03c40224173981d602475e4321c"
format_stamp: "Formatted at 2026-08-12 03:08:56 on dist-test-slave-zpfg"
server_key: "cd62dda9e624bdd2a43cba4b11414296"
server_key_iv: "8cbece2c5f4f18005ceda099bc4b573e"
server_key_version: "encryptionkey@0"
I20260812 03:08:56.393060 5085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance:
uuid: "ec8bf03c40224173981d602475e4321c"
format_stamp: "Formatted at 2026-08-12 03:08:56 on dist-test-slave-zpfg"
server_key: "cd62dda9e624bdd2a43cba4b11414296"
server_key_iv: "8cbece2c5f4f18005ceda099bc4b573e"
server_key_version: "encryptionkey@0"
I20260812 03:08:56.397219 5085 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:08:56.400182 5100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:56.401728 5085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:08:56.401895 5085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "ec8bf03c40224173981d602475e4321c"
format_stamp: "Formatted at 2026-08-12 03:08:56 on dist-test-slave-zpfg"
server_key: "cd62dda9e624bdd2a43cba4b11414296"
server_key_iv: "8cbece2c5f4f18005ceda099bc4b573e"
server_key_version: "encryptionkey@0"
I20260812 03:08:56.402060 5085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:56.422122 5085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:56.422988 5085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:56.423214 5085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:56.423913 5085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:56.424993 5085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:56.425043 5085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:56.425118 5085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:56.425153 5085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:56.434588 5085 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:41001
I20260812 03:08:56.434612 5216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:41001 every 8 connection(s)
I20260812 03:08:56.435701 5085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:08:56.443858 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 5085
I20260812 03:08:56.444001 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance
I20260812 03:08:56.444308 27626 external_mini_cluster.cc:1428] Setting key e748f783cc0e97f88e1690613b6b68bc
I20260812 03:08:56.445917 5064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:56.437727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:34115 (local address 127.26.250.190:36311)
0812 03:08:56.437878 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:56.437882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:56.439072 (+ 1190us) server_negotiation.cc:407] Connection header received
0812 03:08:56.440517 (+ 1445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:56.440519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:56.440574 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:56.440654 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:56.441670 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:56.442251 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:56.443593 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:56.443849 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:56.443978 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:56.445360 (+ 1382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:56.445364 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:56.445366 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:08:56.445378 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:56.445398 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:08:56.445407 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:56.445463 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:56.445656 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 03:08:56.445748 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:08:56.446786 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:0
--local_ip_for_outbound_sockets=127.26.250.130
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:36311
--builtin_ntp_servers=127.26.250.148:34753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 03:08:56.447196 5219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:56.438117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:36311 (local address 127.26.250.129:34115)
0812 03:08:56.438904 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:56.438951 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:56.440297 (+ 1346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:56.440838 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:56.440846 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:56.440933 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:56.441521 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:56.441536 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:56.442443 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:56.442446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:56.443425 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:56.443435 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:56.444040 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:56.444045 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:56.444233 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:56.445013 (+ 780us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:08:56.445015 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:56.445065 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:08:56.445165 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:56.445495 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:56.445508 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:56.445514 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:08:56.445655 (+ 141us) client_negotiation.cc:240] Negotiation successful
0812 03:08:56.446030 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":206,"threads_started":1}
I20260812 03:08:56.448596 5217 heartbeater.cc:347] Connected to a master server at 127.26.250.190:36311
I20260812 03:08:56.448874 5217 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:56.449460 5217 heartbeater.cc:511] Master 127.26.250.190:36311 requested a full tablet report, sending...
I20260812 03:08:56.451084 5023 ts_manager.cc:194] Registered new tserver with Master: ec8bf03c40224173981d602475e4321c (127.26.250.129:41001)
I20260812 03:08:56.452292 5023 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.250.129:34115
W20260812 03:08:56.570211 5220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:56.570605 5220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:56.570668 5220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:56.574792 5220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:56.574888 5220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:56.574981 5220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:08:56.579957 5220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:34753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:36311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:56.581207 5220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:56.582700 5220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:56.593194 5225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:56.593398 5226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:56.593487 5220 server_base.cc:1034] running on GCE node
W20260812 03:08:56.593247 5228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:56.594182 5220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:56.594877 5220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:56.596045 5220 hybrid_clock.cc:648] HybridClock initialized: now 1786504136596023 us; error 36 us; skew 500 ppm
I20260812 03:08:56.597950 5220 webserver.cc:460] Webserver started at http://127.26.250.130:46573/ using document root <none> and password file <none>
I20260812 03:08:56.598594 5220 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:56.598661 5220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:56.598853 5220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:56.600945 5220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/data/instance:
uuid: "11d227d86c624031af0ff5634449fca0"
format_stamp: "Formatted at 2026-08-12 03:08:56 on dist-test-slave-zpfg"
server_key: "a2857d0176a1f01126f8438d57a443c5"
server_key_iv: "f2464da6aeca1b3d91045ed3f8882a5a"
server_key_version: "encryptionkey@0"
I20260812 03:08:56.601477 5220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance:
uuid: "11d227d86c624031af0ff5634449fca0"
format_stamp: "Formatted at 2026-08-12 03:08:56 on dist-test-slave-zpfg"
server_key: "a2857d0176a1f01126f8438d57a443c5"
server_key_iv: "f2464da6aeca1b3d91045ed3f8882a5a"
server_key_version: "encryptionkey@0"
I20260812 03:08:56.605373 5220 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:08:56.608371 5234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:56.609664 5220 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 03:08:56.609767 5220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "11d227d86c624031af0ff5634449fca0"
format_stamp: "Formatted at 2026-08-12 03:08:56 on dist-test-slave-zpfg"
server_key: "a2857d0176a1f01126f8438d57a443c5"
server_key_iv: "f2464da6aeca1b3d91045ed3f8882a5a"
server_key_version: "encryptionkey@0"
I20260812 03:08:56.609881 5220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:56.624437 5220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:56.625145 5220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:56.625321 5220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:56.625886 5220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:56.626922 5220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:56.626974 5220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:56.627014 5220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:56.627030 5220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:56.637084 5220 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:32789
I20260812 03:08:56.637118 5348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:32789 every 8 connection(s)
I20260812 03:08:56.638091 5220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:08:56.644971 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 5220
I20260812 03:08:56.645123 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance
I20260812 03:08:56.645444 27626 external_mini_cluster.cc:1428] Setting key 88af572b5c8bda3b0cd269a77d8e69ef
I20260812 03:08:56.647980 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:0
--local_ip_for_outbound_sockets=127.26.250.131
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:36311
--builtin_ntp_servers=127.26.250.148:34753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 03:08:56.648720 5064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:56.640118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:56423 (local address 127.26.250.190:36311)
0812 03:08:56.640406 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:56.640410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:56.641504 (+ 1094us) server_negotiation.cc:407] Connection header received
0812 03:08:56.643084 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:56.643087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:56.643158 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:56.643242 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:56.644235 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:56.644775 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:56.646276 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:56.646682 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:56.646801 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:56.648167 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:56.648171 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:56.648175 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:08:56.648189 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:56.648219 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:08:56.648227 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:56.648319 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:56.648488 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 03:08:56.648564 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168}
I20260812 03:08:56.649719 5351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:56.640588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:36311 (local address 127.26.250.130:56423)
0812 03:08:56.641303 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:56.641363 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:56.642828 (+ 1465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:56.643436 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:56.643443 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:56.643536 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:56.644068 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:56.644082 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:56.644917 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:56.644925 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:56.646146 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:56.646158 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:56.646865 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:56.646870 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:56.647108 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:56.647866 (+ 758us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:08:56.647868 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:56.647915 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:08:56.647992 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:56.648315 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:56.648328 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:56.648337 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 03:08:56.648488 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 03:08:56.648869 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":116,"threads_started":1}
I20260812 03:08:56.651104 5349 heartbeater.cc:347] Connected to a master server at 127.26.250.190:36311
I20260812 03:08:56.651408 5349 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:56.652127 5349 heartbeater.cc:511] Master 127.26.250.190:36311 requested a full tablet report, sending...
I20260812 03:08:56.653512 5023 ts_manager.cc:194] Registered new tserver with Master: 11d227d86c624031af0ff5634449fca0 (127.26.250.130:32789)
I20260812 03:08:56.654191 5023 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.250.130:56423
W20260812 03:08:56.769368 5352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:56.769702 5352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:56.769778 5352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:56.774750 5352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:56.774854 5352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:56.774976 5352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:08:56.782742 5352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:34753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:36311
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:56.784278 5352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:56.785735 5352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:56.796648 5357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:56.796744 5352 server_base.cc:1034] running on GCE node
W20260812 03:08:56.796648 5358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:56.796720 5360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:56.797420 5352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:56.798133 5352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:56.799337 5352 hybrid_clock.cc:648] HybridClock initialized: now 1786504136799318 us; error 42 us; skew 500 ppm
I20260812 03:08:56.801318 5352 webserver.cc:460] Webserver started at http://127.26.250.131:40933/ using document root <none> and password file <none>
I20260812 03:08:56.801991 5352 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:56.802093 5352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:56.802389 5352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:56.804697 5352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/data/instance:
uuid: "dadda9039eac4cefb0b4935d697b0294"
format_stamp: "Formatted at 2026-08-12 03:08:56 on dist-test-slave-zpfg"
server_key: "dff59c05b8419232e8e1571b4ebfbd3d"
server_key_iv: "a5d045f6665066cf0533dbb73a68f06a"
server_key_version: "encryptionkey@0"
I20260812 03:08:56.805385 5352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance:
uuid: "dadda9039eac4cefb0b4935d697b0294"
format_stamp: "Formatted at 2026-08-12 03:08:56 on dist-test-slave-zpfg"
server_key: "dff59c05b8419232e8e1571b4ebfbd3d"
server_key_iv: "a5d045f6665066cf0533dbb73a68f06a"
server_key_version: "encryptionkey@0"
I20260812 03:08:56.809574 5352 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 03:08:56.812569 5366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:56.814123 5352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:08:56.814389 5352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "dadda9039eac4cefb0b4935d697b0294"
format_stamp: "Formatted at 2026-08-12 03:08:56 on dist-test-slave-zpfg"
server_key: "dff59c05b8419232e8e1571b4ebfbd3d"
server_key_iv: "a5d045f6665066cf0533dbb73a68f06a"
server_key_version: "encryptionkey@0"
I20260812 03:08:56.814525 5352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:56.831794 5352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:56.832549 5352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:56.832718 5352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:56.833331 5352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:56.834465 5352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:56.834535 5352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:56.834607 5352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:56.834652 5352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:56.844259 5352 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:39513
I20260812 03:08:56.844334 5485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:39513 every 8 connection(s)
I20260812 03:08:56.845373 5352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:08:56.847100 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 5352
I20260812 03:08:56.847204 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance
I20260812 03:08:56.847599 27626 external_mini_cluster.cc:1428] Setting key f5dfb62f926bb818c2cb7d3164959717
I20260812 03:08:56.855973 5064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:56.847537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:46631 (local address 127.26.250.190:36311)
0812 03:08:56.847712 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:56.847717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:56.849225 (+ 1508us) server_negotiation.cc:407] Connection header received
0812 03:08:56.850924 (+ 1699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:56.850927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:56.850973 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:56.851047 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:56.852052 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:56.852574 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:56.853899 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:56.854108 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:56.854187 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:56.855552 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:56.855555 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:56.855557 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:08:56.855569 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:56.855589 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:08:56.855596 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:56.855656 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:56.855795 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 03:08:56.855848 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 03:08:56.857043 5488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:56.848094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:36311 (local address 127.26.250.131:46631)
0812 03:08:56.849047 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:56.849110 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:56.850722 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:56.851251 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:56.851258 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:56.851353 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:56.851905 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:56.851919 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:56.852766 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:56.852769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:56.853776 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:56.853789 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:56.854253 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:56.854256 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:56.854570 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:56.855295 (+ 725us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:08:56.855297 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:56.855338 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:08:56.855407 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:56.855671 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:56.855679 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:56.855683 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:08:56.855783 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 03:08:56.856020 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":166,"threads_started":1}
I20260812 03:08:56.858290 5486 heartbeater.cc:347] Connected to a master server at 127.26.250.190:36311
I20260812 03:08:56.858588 5486 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:56.859100 5486 heartbeater.cc:511] Master 127.26.250.190:36311 requested a full tablet report, sending...
I20260812 03:08:56.860157 5023 ts_manager.cc:194] Registered new tserver with Master: dadda9039eac4cefb0b4935d697b0294 (127.26.250.131:39513)
I20260812 03:08:56.860843 5023 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.250.131:46631
I20260812 03:08:56.862066 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 03:08:56.865242 5064 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:56.864272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59362 (local address 127.26.250.190:36311)
0812 03:08:56.864423 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:56.864428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:56.864613 (+ 185us) server_negotiation.cc:407] Connection header received
0812 03:08:56.864712 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:56.864714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:56.864755 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:56.864826 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:56.865106 (+ 280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59362: BlockingRecv error: recv got EOF from 127.0.0.1:59362 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 03:08:56.865787 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 5085
I20260812 03:08:56.878891 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 5220
I20260812 03:08:56.887321 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 5352
I20260812 03:08:56.896432 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 4989
2026-08-12T03:08:56Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (961 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T03:08:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:08:56Z Disabled control of system clock
I20260812 03:08:56.931121 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:32817
--webserver_interface=127.26.250.190
--webserver_port=0
--builtin_ntp_servers=127.26.250.148:34651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:32817
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 03:08:57.051651 5497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:57.051976 5497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:57.052028 5497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:57.056535 5497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:08:57.056602 5497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:57.056619 5497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:57.056639 5497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:08:57.056656 5497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:08:57.062444 5497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:34651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:32817
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:32817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:57.063840 5497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:57.065475 5497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:57.075949 5507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:57.075934 5502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:57.075942 5503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:57.076409 5497 server_base.cc:1034] running on GCE node
I20260812 03:08:57.077031 5497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:57.078249 5497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:57.079447 5497 hybrid_clock.cc:648] HybridClock initialized: now 1786504137079414 us; error 49 us; skew 500 ppm
I20260812 03:08:57.081550 5497 webserver.cc:460] Webserver started at http://127.26.250.190:42597/ using document root <none> and password file <none>
I20260812 03:08:57.082175 5497 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:57.082275 5497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:57.082541 5497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:57.084723 5497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "4c71ab9d568b485c88be6aeec4429509"
format_stamp: "Formatted at 2026-08-12 03:08:57 on dist-test-slave-zpfg"
server_key: "2ad56e8afc2311034355bdcc972c58c6"
server_key_iv: "b721cea3e89126c52c976d7b318a95f8"
server_key_version: "encryptionkey@0"
I20260812 03:08:57.085355 5497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "4c71ab9d568b485c88be6aeec4429509"
format_stamp: "Formatted at 2026-08-12 03:08:57 on dist-test-slave-zpfg"
server_key: "2ad56e8afc2311034355bdcc972c58c6"
server_key_iv: "b721cea3e89126c52c976d7b318a95f8"
server_key_version: "encryptionkey@0"
I20260812 03:08:57.089395 5497 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 03:08:57.092203 5513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:57.094043 5497 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:08:57.094249 5497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "4c71ab9d568b485c88be6aeec4429509"
format_stamp: "Formatted at 2026-08-12 03:08:57 on dist-test-slave-zpfg"
server_key: "2ad56e8afc2311034355bdcc972c58c6"
server_key_iv: "b721cea3e89126c52c976d7b318a95f8"
server_key_version: "encryptionkey@0"
I20260812 03:08:57.094481 5497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:57.100112 5497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:57.100898 5497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:57.101125 5497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:57.109879 5566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:32817 every 8 connection(s)
I20260812 03:08:57.109875 5497 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:32817
I20260812 03:08:57.111503 5497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:08:57.115526 5567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:57.118939 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 5497
I20260812 03:08:57.119045 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:08:57.119329 27626 external_mini_cluster.cc:1428] Setting key 00ff44a0d6093b29697f97e6bd0672ec
I20260812 03:08:57.122836 5567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c71ab9d568b485c88be6aeec4429509: Bootstrap starting.
I20260812 03:08:57.125821 5570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:57.120988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35462 (local address 127.26.250.190:32817)
0812 03:08:57.121805 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:57.121823 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:57.121864 (+ 41us) server_negotiation.cc:407] Connection header received
0812 03:08:57.122629 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:57.122649 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:57.123105 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:57.123495 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:57.124457 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:57.124462 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:57.124476 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:08:57.124489 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:57.124552 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:08:57.124570 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:57.124735 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:57.124811 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 03:08:57.125066 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":170,"threads_started":1}
I20260812 03:08:57.125844 5567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c71ab9d568b485c88be6aeec4429509: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:57.127014 5567 log.cc:824] T 00000000000000000000000000000000 P 4c71ab9d568b485c88be6aeec4429509: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:57.129552 5567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c71ab9d568b485c88be6aeec4429509: No bootstrap required, opened a new log
I20260812 03:08:57.133985 5567 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c71ab9d568b485c88be6aeec4429509 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c71ab9d568b485c88be6aeec4429509" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 32817 } }
I20260812 03:08:57.134330 5567 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c71ab9d568b485c88be6aeec4429509 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:57.134455 5567 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c71ab9d568b485c88be6aeec4429509 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c71ab9d568b485c88be6aeec4429509, State: Initialized, Role: FOLLOWER
I20260812 03:08:57.134975 5567 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c71ab9d568b485c88be6aeec4429509 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c71ab9d568b485c88be6aeec4429509" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 32817 } }
I20260812 03:08:57.135154 5567 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c71ab9d568b485c88be6aeec4429509 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:08:57.135267 5567 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c71ab9d568b485c88be6aeec4429509 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:08:57.135409 5567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c71ab9d568b485c88be6aeec4429509 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:57.136615 5567 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c71ab9d568b485c88be6aeec4429509 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c71ab9d568b485c88be6aeec4429509" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 32817 } }
I20260812 03:08:57.137022 5567 leader_election.cc:302] T 00000000000000000000000000000000 P 4c71ab9d568b485c88be6aeec4429509 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c71ab9d568b485c88be6aeec4429509; no voters:
I20260812 03:08:57.137491 5567 leader_election.cc:288] T 00000000000000000000000000000000 P 4c71ab9d568b485c88be6aeec4429509 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:08:57.137624 5572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c71ab9d568b485c88be6aeec4429509 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:57.137912 5572 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c71ab9d568b485c88be6aeec4429509 [term 1 LEADER]: Becoming Leader. State: Replica: 4c71ab9d568b485c88be6aeec4429509, State: Running, Role: LEADER
I20260812 03:08:57.138238 5572 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c71ab9d568b485c88be6aeec4429509 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c71ab9d568b485c88be6aeec4429509" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 32817 } }
I20260812 03:08:57.139038 5567 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c71ab9d568b485c88be6aeec4429509 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:08:57.140093 5573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c71ab9d568b485c88be6aeec4429509 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c71ab9d568b485c88be6aeec4429509" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c71ab9d568b485c88be6aeec4429509" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 32817 } } }
I20260812 03:08:57.140228 5573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c71ab9d568b485c88be6aeec4429509 [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:57.140439 5574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c71ab9d568b485c88be6aeec4429509 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c71ab9d568b485c88be6aeec4429509. Latest consensus state: current_term: 1 leader_uuid: "4c71ab9d568b485c88be6aeec4429509" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c71ab9d568b485c88be6aeec4429509" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 32817 } } }
I20260812 03:08:57.140532 5574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c71ab9d568b485c88be6aeec4429509 [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:57.140812 5579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:08:57.144375 5579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:08:57.150972 5579 catalog_manager.cc:1347] Generated new cluster ID: fb63d286e7044ffaa4ad16a4d8830218
I20260812 03:08:57.151064 5579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:08:57.184525 5579 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:08:57.184931 5579 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:08:57.193392 5579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c71ab9d568b485c88be6aeec4429509: Generated new TSK 0
I20260812 03:08:57.194170 5579 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:08:57.206748 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:0
--local_ip_for_outbound_sockets=127.26.250.129
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:32817
--builtin_ntp_servers=127.26.250.148:34651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 03:08:57.356873 5595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:57.357160 5595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:57.357200 5595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:57.361603 5595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:57.361681 5595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:57.361742 5595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:08:57.366629 5595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:34651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:32817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:57.367786 5595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:57.369212 5595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:57.379711 5601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:57.379805 5595 server_base.cc:1034] running on GCE node
W20260812 03:08:57.379711 5600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:57.379711 5603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:57.380436 5595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:57.381146 5595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:57.382316 5595 hybrid_clock.cc:648] HybridClock initialized: now 1786504137382301 us; error 31 us; skew 500 ppm
I20260812 03:08:57.384375 5595 webserver.cc:460] Webserver started at http://127.26.250.129:40297/ using document root <none> and password file <none>
I20260812 03:08:57.385006 5595 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:57.385106 5595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:57.385349 5595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:57.387624 5595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/data/instance:
uuid: "7d22abacbbeb44bfb037836cc3f26f79"
format_stamp: "Formatted at 2026-08-12 03:08:57 on dist-test-slave-zpfg"
server_key: "804c0f2a60edf1d97ad783873ff551e6"
server_key_iv: "ad4a70c90950aca381ba5e2347b6b065"
server_key_version: "encryptionkey@0"
I20260812 03:08:57.388257 5595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance:
uuid: "7d22abacbbeb44bfb037836cc3f26f79"
format_stamp: "Formatted at 2026-08-12 03:08:57 on dist-test-slave-zpfg"
server_key: "804c0f2a60edf1d97ad783873ff551e6"
server_key_iv: "ad4a70c90950aca381ba5e2347b6b065"
server_key_version: "encryptionkey@0"
I20260812 03:08:57.392261 5595 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:08:57.395170 5609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:57.396579 5595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:08:57.396724 5595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "7d22abacbbeb44bfb037836cc3f26f79"
format_stamp: "Formatted at 2026-08-12 03:08:57 on dist-test-slave-zpfg"
server_key: "804c0f2a60edf1d97ad783873ff551e6"
server_key_iv: "ad4a70c90950aca381ba5e2347b6b065"
server_key_version: "encryptionkey@0"
I20260812 03:08:57.396863 5595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:57.401760 5595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:57.402526 5595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:57.402721 5595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:57.403306 5595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:57.404268 5595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:57.404340 5595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:57.404407 5595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:57.404450 5595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:57.414032 5595 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:32941
I20260812 03:08:57.414074 5726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:32941 every 8 connection(s)
I20260812 03:08:57.415021 5595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:08:57.415448 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 5595
I20260812 03:08:57.415578 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance
I20260812 03:08:57.416029 27626 external_mini_cluster.cc:1428] Setting key aa6625004ac7dbf350fda9ad15df7bcc
I20260812 03:08:57.418467 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:0
--local_ip_for_outbound_sockets=127.26.250.130
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:32817
--builtin_ntp_servers=127.26.250.148:34651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 03:08:57.421716 5570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:57.416899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:46565 (local address 127.26.250.190:32817)
0812 03:08:57.417033 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:57.417037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:57.418328 (+ 1291us) server_negotiation.cc:407] Connection header received
0812 03:08:57.419798 (+ 1470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:57.419802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:57.419870 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:57.419950 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:57.421256 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:57.421260 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:57.421262 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:08:57.421274 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:57.421296 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:08:57.421305 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:57.421394 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:57.421566 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 03:08:57.421606 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 03:08:57.422799 5731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:57.417457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:32817 (local address 127.26.250.129:46565)
0812 03:08:57.418184 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:57.418226 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:57.419588 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:57.420134 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:57.420146 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:57.420273 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:57.420897 (+ 624us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:08:57.420900 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:57.420971 (+ 71us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:08:57.421070 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:57.421406 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:57.421417 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:57.421424 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 03:08:57.421609 (+ 185us) client_negotiation.cc:240] Negotiation successful
0812 03:08:57.421860 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":114,"threads_started":1}
I20260812 03:08:57.424263 5728 heartbeater.cc:347] Connected to a master server at 127.26.250.190:32817
I20260812 03:08:57.424530 5728 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:57.424973 5728 heartbeater.cc:511] Master 127.26.250.190:32817 requested a full tablet report, sending...
I20260812 03:08:57.426602 5530 ts_manager.cc:194] Registered new tserver with Master: 7d22abacbbeb44bfb037836cc3f26f79 (127.26.250.129:32941)
W20260812 03:08:57.540355 5732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:57.540707 5732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:57.540755 5732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:57.545074 5732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:57.545151 5732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:57.545212 5732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:08:57.550191 5732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:34651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:32817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:57.551344 5732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:57.552723 5732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:57.563217 5738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:57.563225 5737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:57.563225 5740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:57.563576 5732 server_base.cc:1034] running on GCE node
I20260812 03:08:57.564127 5732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:57.564843 5732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:57.566049 5732 hybrid_clock.cc:648] HybridClock initialized: now 1786504137566033 us; error 30 us; skew 500 ppm
I20260812 03:08:57.568387 5732 webserver.cc:460] Webserver started at http://127.26.250.130:46205/ using document root <none> and password file <none>
I20260812 03:08:57.569031 5732 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:57.569099 5732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:57.569336 5732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:57.571944 5732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/data/instance:
uuid: "87e4e97939914bcfba51684686fca29c"
format_stamp: "Formatted at 2026-08-12 03:08:57 on dist-test-slave-zpfg"
server_key: "8b0547bc1aef1b638159c151616b4b92"
server_key_iv: "895ebfae03c084f22a38dd6d9ba102d2"
server_key_version: "encryptionkey@0"
I20260812 03:08:57.572618 5732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance:
uuid: "87e4e97939914bcfba51684686fca29c"
format_stamp: "Formatted at 2026-08-12 03:08:57 on dist-test-slave-zpfg"
server_key: "8b0547bc1aef1b638159c151616b4b92"
server_key_iv: "895ebfae03c084f22a38dd6d9ba102d2"
server_key_version: "encryptionkey@0"
I20260812 03:08:57.577195 5732 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:08:57.580286 5747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:57.581938 5732 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:08:57.582122 5732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "87e4e97939914bcfba51684686fca29c"
format_stamp: "Formatted at 2026-08-12 03:08:57 on dist-test-slave-zpfg"
server_key: "8b0547bc1aef1b638159c151616b4b92"
server_key_iv: "895ebfae03c084f22a38dd6d9ba102d2"
server_key_version: "encryptionkey@0"
I20260812 03:08:57.582271 5732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:57.587461 5732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:57.588140 5732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:57.588340 5732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:57.588922 5732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:57.589893 5732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:57.589967 5732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:57.590039 5732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:57.590091 5732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:57.600390 5732 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:40415
I20260812 03:08:57.600409 5868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:40415 every 8 connection(s)
I20260812 03:08:57.601482 5732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:08:57.607999 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 5732
I20260812 03:08:57.608166 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance
I20260812 03:08:57.608494 27626 external_mini_cluster.cc:1428] Setting key a12f6d9630c53149ab73eb7b4b4161b8
I20260812 03:08:57.608558 5570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:57.603482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:35179 (local address 127.26.250.190:32817)
0812 03:08:57.603662 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:57.603667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:57.604968 (+ 1301us) server_negotiation.cc:407] Connection header received
0812 03:08:57.606434 (+ 1466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:57.606438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:57.606492 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:57.606600 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:57.607867 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:57.607871 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:57.607874 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:08:57.607891 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:57.607917 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:08:57.607929 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:57.608042 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:57.608358 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 03:08:57.608412 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 03:08:57.609598 5871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:57.603923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:32817 (local address 127.26.250.130:35179)
0812 03:08:57.604792 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:57.604842 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:57.606119 (+ 1277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:57.606841 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:57.606849 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:57.606947 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:57.607540 (+ 593us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:08:57.607542 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:57.607587 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:08:57.607681 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:57.608029 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:57.608038 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:57.608042 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:08:57.608361 (+ 319us) client_negotiation.cc:240] Negotiation successful
0812 03:08:57.608625 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":130,"threads_started":1}
I20260812 03:08:57.610880 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:0
--local_ip_for_outbound_sockets=127.26.250.131
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:32817
--builtin_ntp_servers=127.26.250.148:34651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 03:08:57.610930 5869 heartbeater.cc:347] Connected to a master server at 127.26.250.190:32817
I20260812 03:08:57.611191 5869 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:57.611671 5869 heartbeater.cc:511] Master 127.26.250.190:32817 requested a full tablet report, sending...
I20260812 03:08:57.612866 5530 ts_manager.cc:194] Registered new tserver with Master: 87e4e97939914bcfba51684686fca29c (127.26.250.130:40415)
W20260812 03:08:57.737169 5872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:57.737466 5872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:57.737509 5872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:57.741834 5872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:57.741916 5872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:57.741978 5872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:08:57.746879 5872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:34651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:32817
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:57.748011 5872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:57.749552 5872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:57.759809 5878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:57.759838 5881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:57.759946 5872 server_base.cc:1034] running on GCE node
W20260812 03:08:57.759887 5879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:57.760553 5872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:57.761265 5872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:57.762465 5872 hybrid_clock.cc:648] HybridClock initialized: now 1786504137762447 us; error 29 us; skew 500 ppm
I20260812 03:08:57.764811 5872 webserver.cc:460] Webserver started at http://127.26.250.131:44257/ using document root <none> and password file <none>
I20260812 03:08:57.765455 5872 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:57.765556 5872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:57.765801 5872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:57.768026 5872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/data/instance:
uuid: "9b0a3fb2133a465e8a03b9428d245876"
format_stamp: "Formatted at 2026-08-12 03:08:57 on dist-test-slave-zpfg"
server_key: "e24d4bf6785ebb6977d662bb11bcc165"
server_key_iv: "70b8b6ecd418ea826a44ef9bd67cd063"
server_key_version: "encryptionkey@0"
I20260812 03:08:57.768633 5872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance:
uuid: "9b0a3fb2133a465e8a03b9428d245876"
format_stamp: "Formatted at 2026-08-12 03:08:57 on dist-test-slave-zpfg"
server_key: "e24d4bf6785ebb6977d662bb11bcc165"
server_key_iv: "70b8b6ecd418ea826a44ef9bd67cd063"
server_key_version: "encryptionkey@0"
I20260812 03:08:57.772953 5872 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:08:57.776011 5887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:57.777385 5872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:08:57.777558 5872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "9b0a3fb2133a465e8a03b9428d245876"
format_stamp: "Formatted at 2026-08-12 03:08:57 on dist-test-slave-zpfg"
server_key: "e24d4bf6785ebb6977d662bb11bcc165"
server_key_iv: "70b8b6ecd418ea826a44ef9bd67cd063"
server_key_version: "encryptionkey@0"
I20260812 03:08:57.777696 5872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:57.782691 5872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:57.783396 5872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:57.783593 5872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:57.784173 5872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:57.785181 5872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:57.785250 5872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:57.785320 5872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:57.785373 5872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:57.794749 5872 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:39165
I20260812 03:08:57.794787 6007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:39165 every 8 connection(s)
I20260812 03:08:57.795730 5872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:08:57.799387 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 5872
I20260812 03:08:57.799563 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance
I20260812 03:08:57.799886 27626 external_mini_cluster.cc:1428] Setting key c86761dc527491435dfc48913b96eb4f
I20260812 03:08:57.803517 5570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:57.797783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:49665 (local address 127.26.250.190:32817)
0812 03:08:57.797950 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:57.797955 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:57.799498 (+ 1543us) server_negotiation.cc:407] Connection header received
0812 03:08:57.801360 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:57.801366 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:57.801417 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:57.801506 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:57.803024 (+ 1518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:57.803027 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:57.803029 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:08:57.803041 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:57.803062 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:08:57.803070 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:57.803155 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:57.803352 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 03:08:57.803400 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 03:08:57.804493 6010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:57.798394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:32817 (local address 127.26.250.131:49665)
0812 03:08:57.799322 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:57.799368 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:57.801114 (+ 1746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:57.801769 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:57.801780 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:57.801957 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:57.802725 (+ 768us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:08:57.802727 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:57.802773 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:08:57.802853 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:57.803158 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:57.803165 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:57.803169 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:08:57.803316 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 03:08:57.803538 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":164,"threads_started":1}
I20260812 03:08:57.805931 6008 heartbeater.cc:347] Connected to a master server at 127.26.250.190:32817
I20260812 03:08:57.806218 6008 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:57.806726 6008 heartbeater.cc:511] Master 127.26.250.190:32817 requested a full tablet report, sending...
I20260812 03:08:57.807940 5530 ts_manager.cc:194] Registered new tserver with Master: 9b0a3fb2133a465e8a03b9428d245876 (127.26.250.131:39165)
I20260812 03:08:57.815032 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 03:08:57.818624 5570 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:08:57.817393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48784 (local address 127.26.250.190:32817)
0812 03:08:57.817555 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:57.817559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:57.817774 (+ 215us) server_negotiation.cc:407] Connection header received
0812 03:08:57.817973 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:57.817979 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:57.818027 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:57.818126 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:57.818456 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48784: BlockingRecv error: recv got EOF from 127.0.0.1:48784 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 03:08:57.818892 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 5595
I20260812 03:08:57.827266 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 5732
I20260812 03:08:57.836040 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 5872
I20260812 03:08:57.844780 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 5497
2026-08-12T03:08:57Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (947 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:08:57 dist-test-slave-zpfg krb5kdc[6019](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:08:57 dist-test-slave-zpfg krb5kdc[6019](info): set up 2 sockets
Aug 12 03:08:57 dist-test-slave-zpfg krb5kdc[6019](info): commencing operation
krb5kdc: starting...
W20260812 03:08:58.960917 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:08:59 dist-test-slave-zpfg krb5kdc[6019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:08:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:08:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:08:59.110045 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:35937
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:44935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:35937
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:59.247607 6033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:59.247963 6033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:59.248066 6033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:59.252763 6033 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:08:59.252871 6033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:59.252916 6033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:59.252955 6033 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:08:59.252993 6033 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:08:59.259063 6033 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:44935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:35937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:35937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:59.260332 6033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:59.261503 6033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:59.271432 6039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:59.271417 6038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:59.271417 6042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:59.271824 6033 server_base.cc:1034] running on GCE node
I20260812 03:08:59.272593 6033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:59.273912 6033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:59.275134 6033 hybrid_clock.cc:648] HybridClock initialized: now 1786504139275115 us; error 45 us; skew 500 ppm
Aug 12 03:08:59 dist-test-slave-zpfg krb5kdc[6019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504139, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:59.279050 6033 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:08:59.280193 6033 webserver.cc:460] Webserver started at http://127.26.250.190:37313/ using document root <none> and password file <none>
I20260812 03:08:59.280831 6033 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:59.280933 6033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:59.281190 6033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:59.283551 6033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "2ac4e629553443cfaa1984eaa8ed8228"
format_stamp: "Formatted at 2026-08-12 03:08:59 on dist-test-slave-zpfg"
server_key: "f9dbae6e47e1d5fd508ac88e567bc7c1"
server_key_iv: "2baaaceb49cc435258ba33475ad66a82"
server_key_version: "encryptionkey@0"
I20260812 03:08:59.284206 6033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "2ac4e629553443cfaa1984eaa8ed8228"
format_stamp: "Formatted at 2026-08-12 03:08:59 on dist-test-slave-zpfg"
server_key: "f9dbae6e47e1d5fd508ac88e567bc7c1"
server_key_iv: "2baaaceb49cc435258ba33475ad66a82"
server_key_version: "encryptionkey@0"
I20260812 03:08:59.288658 6033 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:08:59.291568 6050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:59.293085 6033 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:08:59.293264 6033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "2ac4e629553443cfaa1984eaa8ed8228"
format_stamp: "Formatted at 2026-08-12 03:08:59 on dist-test-slave-zpfg"
server_key: "f9dbae6e47e1d5fd508ac88e567bc7c1"
server_key_iv: "2baaaceb49cc435258ba33475ad66a82"
server_key_version: "encryptionkey@0"
I20260812 03:08:59.293412 6033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:59.307938 6033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:59.311856 6033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:59.312103 6033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:59.320381 6033 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:35937
I20260812 03:08:59.320525 6104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:35937 every 8 connection(s)
I20260812 03:08:59.321506 6033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:08:59.324855 6105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:08:59.328917 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 6033
I20260812 03:08:59.329073 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:08:59.329380 27626 external_mini_cluster.cc:1428] Setting key d3f184446dcbffd77aa0e2a47c51edeb
I20260812 03:08:59.331410 6105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ac4e629553443cfaa1984eaa8ed8228: Bootstrap starting.
I20260812 03:08:59.334196 6105 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ac4e629553443cfaa1984eaa8ed8228: Neither blocks nor log segments found. Creating new log.
I20260812 03:08:59.335057 6105 log.cc:824] T 00000000000000000000000000000000 P 2ac4e629553443cfaa1984eaa8ed8228: Log is configured to *not* fsync() on all Append() calls
I20260812 03:08:59.337548 6105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ac4e629553443cfaa1984eaa8ed8228: No bootstrap required, opened a new log
Aug 12 03:08:59 dist-test-slave-zpfg krb5kdc[6019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:59.341753 6105 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ac4e629553443cfaa1984eaa8ed8228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac4e629553443cfaa1984eaa8ed8228" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 35937 } }
I20260812 03:08:59.342056 6105 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ac4e629553443cfaa1984eaa8ed8228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:08:59.342118 6105 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ac4e629553443cfaa1984eaa8ed8228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ac4e629553443cfaa1984eaa8ed8228, State: Initialized, Role: FOLLOWER
I20260812 03:08:59.342685 6105 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ac4e629553443cfaa1984eaa8ed8228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac4e629553443cfaa1984eaa8ed8228" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 35937 } }
I20260812 03:08:59.342864 6105 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ac4e629553443cfaa1984eaa8ed8228 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:08:59.342954 6105 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ac4e629553443cfaa1984eaa8ed8228 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:08:59.343092 6105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ac4e629553443cfaa1984eaa8ed8228 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:08:59.344285 6105 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ac4e629553443cfaa1984eaa8ed8228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac4e629553443cfaa1984eaa8ed8228" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 35937 } }
I20260812 03:08:59.344640 6105 leader_election.cc:302] T 00000000000000000000000000000000 P 2ac4e629553443cfaa1984eaa8ed8228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ac4e629553443cfaa1984eaa8ed8228; no voters:
I20260812 03:08:59.344990 6105 leader_election.cc:288] T 00000000000000000000000000000000 P 2ac4e629553443cfaa1984eaa8ed8228 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:08:59.345112 6111 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ac4e629553443cfaa1984eaa8ed8228 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:08:59.345328 6111 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ac4e629553443cfaa1984eaa8ed8228 [term 1 LEADER]: Becoming Leader. State: Replica: 2ac4e629553443cfaa1984eaa8ed8228, State: Running, Role: LEADER
I20260812 03:08:59.345397 6109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:59.330964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45828 (local address 127.26.250.190:35937)
0812 03:08:59.331432 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:59.331448 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:59.331488 (+ 40us) server_negotiation.cc:407] Connection header received
0812 03:08:59.332208 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:59.332227 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:59.332588 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:59.332892 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:59.333869 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:59.334996 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:59.336062 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:59.336651 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:59.336792 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:59.339905 (+ 3113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:59.339911 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:59.339926 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:59.339956 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:59.341813 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:59.342475 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:59.342483 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:59.342490 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:59.342620 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:59.342929 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:59.342932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:59.342934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:59.343386 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:59.343565 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:59.343889 (+ 324us) server_negotiation.cc:299] Negotiation successful
0812 03:08:59.344215 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":148,"threads_started":1}
I20260812 03:08:59.345664 6111 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ac4e629553443cfaa1984eaa8ed8228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac4e629553443cfaa1984eaa8ed8228" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 35937 } }
I20260812 03:08:59.346328 6105 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ac4e629553443cfaa1984eaa8ed8228 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:08:59.346939 6113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ac4e629553443cfaa1984eaa8ed8228 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ac4e629553443cfaa1984eaa8ed8228. Latest consensus state: current_term: 1 leader_uuid: "2ac4e629553443cfaa1984eaa8ed8228" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac4e629553443cfaa1984eaa8ed8228" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 35937 } } }
I20260812 03:08:59.347149 6113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ac4e629553443cfaa1984eaa8ed8228 [sys.catalog]: This master's current role is: LEADER
I20260812 03:08:59.347563 6112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ac4e629553443cfaa1984eaa8ed8228 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ac4e629553443cfaa1984eaa8ed8228" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac4e629553443cfaa1984eaa8ed8228" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 35937 } } }
I20260812 03:08:59.347669 6112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ac4e629553443cfaa1984eaa8ed8228 [sys.catalog]: This master's current role is: LEADER
W20260812 03:08:59.351608 6126 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2ac4e629553443cfaa1984eaa8ed8228: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:08:59.351713 6126 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:08:59.351806 6118 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:08:59.352854 6118 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:08:59.359385 6118 catalog_manager.cc:1347] Generated new cluster ID: dfd211664b394ac7980882f965598cdf
I20260812 03:08:59.359498 6118 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:08:59.377813 6118 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:08:59.379016 6118 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:08:59.388880 6118 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ac4e629553443cfaa1984eaa8ed8228: Generated new TSK 0
I20260812 03:08:59.389717 6118 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.250.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
I20260812 03:08:59.426930 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:0
--local_ip_for_outbound_sockets=127.26.250.129
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:35937
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:44935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:59.550422 6132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:59.550724 6132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:59.550770 6132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:59.555052 6132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:59.555130 6132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:59.555220 6132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:08:59.561020 6132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:44935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:35937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:59.562302 6132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:59.563449 6132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:59.574460 6141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:59.574560 6139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:59.574497 6138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:59.574666 6132 server_base.cc:1034] running on GCE node
I20260812 03:08:59.575336 6132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:59.576033 6132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:59.577322 6132 hybrid_clock.cc:648] HybridClock initialized: now 1786504139577202 us; error 62 us; skew 500 ppm
Aug 12 03:08:59 dist-test-slave-zpfg krb5kdc[6019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504139, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:59.580847 6132 init.cc:377] Logged in from keytab as kudu/127.26.250.129@KRBTEST.COM (short username kudu)
I20260812 03:08:59.581887 6132 webserver.cc:460] Webserver started at http://127.26.250.129:44099/ using document root <none> and password file <none>
I20260812 03:08:59.582583 6132 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:59.582653 6132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:59.582890 6132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:59.585100 6132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/data/instance:
uuid: "130ae2c6ec8c40fd81fd8ad81e63fcde"
format_stamp: "Formatted at 2026-08-12 03:08:59 on dist-test-slave-zpfg"
server_key: "d0b0b10afdad8782370fc2e9e68af8b7"
server_key_iv: "3f0be963098ec36a6b2a8df065873abf"
server_key_version: "encryptionkey@0"
I20260812 03:08:59.585726 6132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance:
uuid: "130ae2c6ec8c40fd81fd8ad81e63fcde"
format_stamp: "Formatted at 2026-08-12 03:08:59 on dist-test-slave-zpfg"
server_key: "d0b0b10afdad8782370fc2e9e68af8b7"
server_key_iv: "3f0be963098ec36a6b2a8df065873abf"
server_key_version: "encryptionkey@0"
I20260812 03:08:59.589900 6132 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:08:59.592697 6148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:59.594098 6132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 03:08:59.594247 6132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "130ae2c6ec8c40fd81fd8ad81e63fcde"
format_stamp: "Formatted at 2026-08-12 03:08:59 on dist-test-slave-zpfg"
server_key: "d0b0b10afdad8782370fc2e9e68af8b7"
server_key_iv: "3f0be963098ec36a6b2a8df065873abf"
server_key_version: "encryptionkey@0"
I20260812 03:08:59.594412 6132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:59.611940 6132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:59.616065 6132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:59.616355 6132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:59.617065 6132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:59.618032 6132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:59.618115 6132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:59.618191 6132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:59.618248 6132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:59.628880 6132 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:44869
I20260812 03:08:59.628907 6266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:44869 every 8 connection(s)
I20260812 03:08:59.629964 6132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:08:59.635492 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 6132
I20260812 03:08:59.635649 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance
I20260812 03:08:59.636058 27626 external_mini_cluster.cc:1428] Setting key fa9a9b20d787ada81d25e8c3cca0d29d
WARNING: no policy specified for kudu/127.26.250.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.130@KRBTEST.COM" created.
Aug 12 03:08:59 dist-test-slave-zpfg krb5kdc[6019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504139, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:59.656534 6109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:59.631983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:50855 (local address 127.26.250.190:35937)
0812 03:08:59.632140 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:59.632144 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:59.632987 (+ 843us) server_negotiation.cc:407] Connection header received
0812 03:08:59.635164 (+ 2177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:59.635168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:59.635237 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:59.635364 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:59.637284 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:59.637957 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:59.639863 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:59.640408 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:59.640551 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:59.652249 (+ 11698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:59.652253 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:59.652255 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:59.652291 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:59.653916 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:59.654696 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:59.654699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:59.654700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:59.654786 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:59.655241 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:59.655244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:59.655246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:59.655467 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:59.655584 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:59.656192 (+ 608us) server_negotiation.cc:299] Negotiation successful
0812 03:08:59.656352 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 03:08:59.657445 6269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:59.632309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:35937 (local address 127.26.250.129:50855)
0812 03:08:59.632815 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:59.632864 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:59.634909 (+ 2045us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:59.635568 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:59.635580 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:59.636366 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:59.637065 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:59.637087 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:59.638099 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:59.638104 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:59.639689 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:59.639705 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:59.640592 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:59.640595 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:59.640752 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:59.646465 (+ 5713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:59.646500 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:59.651972 (+ 5472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:59.654091 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:59.654099 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:59.654121 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:59.654555 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:59.654936 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:59.654946 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:59.654947 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:59.655122 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:59.655589 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:59.655599 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:59.655903 (+ 304us) client_negotiation.cc:769] Sending connection context
0812 03:08:59.656111 (+ 208us) client_negotiation.cc:240] Negotiation successful
0812 03:08:59.656514 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":121,"threads_started":1}
I20260812 03:08:59.659276 6267 heartbeater.cc:347] Connected to a master server at 127.26.250.190:35937
I20260812 03:08:59.659601 6267 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:59.660215 6267 heartbeater.cc:511] Master 127.26.250.190:35937 requested a full tablet report, sending...
I20260812 03:08:59.661976 6066 ts_manager.cc:194] Registered new tserver with Master: 130ae2c6ec8c40fd81fd8ad81e63fcde (127.26.250.129:44869)
I20260812 03:08:59.663350 6066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.129@KRBTEST.COM'} at 127.26.250.129:50855
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
I20260812 03:08:59.671898 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:0
--local_ip_for_outbound_sockets=127.26.250.130
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:35937
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:44935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:08:59.795524 6272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:08:59.795823 6272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:08:59.795871 6272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:08:59.800451 6272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:08:59.800523 6272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:08:59.800587 6272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:08:59.806216 6272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:44935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:35937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:08:59.807446 6272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:08:59.808481 6272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:08:59.818974 6280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:59.818988 6278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:08:59.818988 6277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:08:59.819182 6272 server_base.cc:1034] running on GCE node
I20260812 03:08:59.819785 6272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:08:59.820484 6272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:08:59.821647 6272 hybrid_clock.cc:648] HybridClock initialized: now 1786504139821631 us; error 28 us; skew 500 ppm
Aug 12 03:08:59 dist-test-slave-zpfg krb5kdc[6019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504139, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:08:59.824625 6272 init.cc:377] Logged in from keytab as kudu/127.26.250.130@KRBTEST.COM (short username kudu)
I20260812 03:08:59.825660 6272 webserver.cc:460] Webserver started at http://127.26.250.130:35765/ using document root <none> and password file <none>
I20260812 03:08:59.826267 6272 fs_manager.cc:359] Metadata directory not provided
I20260812 03:08:59.826390 6272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:08:59.826638 6272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:08:59.828836 6272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/data/instance:
uuid: "f6da69a2f81b4dbfa59083c70cce002a"
format_stamp: "Formatted at 2026-08-12 03:08:59 on dist-test-slave-zpfg"
server_key: "ec7938ce247310989dccd0c698170c21"
server_key_iv: "2fdac257a7027d75ce01740a4910e837"
server_key_version: "encryptionkey@0"
I20260812 03:08:59.829463 6272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance:
uuid: "f6da69a2f81b4dbfa59083c70cce002a"
format_stamp: "Formatted at 2026-08-12 03:08:59 on dist-test-slave-zpfg"
server_key: "ec7938ce247310989dccd0c698170c21"
server_key_iv: "2fdac257a7027d75ce01740a4910e837"
server_key_version: "encryptionkey@0"
I20260812 03:08:59.833554 6272 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 03:08:59.836436 6288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:59.837774 6272 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 03:08:59.837929 6272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "f6da69a2f81b4dbfa59083c70cce002a"
format_stamp: "Formatted at 2026-08-12 03:08:59 on dist-test-slave-zpfg"
server_key: "ec7938ce247310989dccd0c698170c21"
server_key_iv: "2fdac257a7027d75ce01740a4910e837"
server_key_version: "encryptionkey@0"
I20260812 03:08:59.838059 6272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:08:59.849112 6272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:08:59.852407 6272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:08:59.852622 6272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:08:59.853263 6272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:08:59.854125 6272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:08:59.854208 6272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:08:59.854301 6272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:08:59.854377 6272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260812 03:08:59.864146 6272 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:46167
I20260812 03:08:59.864297 6409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:46167 every 8 connection(s)
I20260812 03:08:59.865134 6272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:08:59.870731 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 6272
I20260812 03:08:59.870872 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance
I20260812 03:08:59.871194 27626 external_mini_cluster.cc:1428] Setting key c65312e40e593ab2b7e6faecb23d260b
Aug 12 03:08:59 dist-test-slave-zpfg krb5kdc[6019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504139, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:08:59.881395 6109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:59.867283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:48751 (local address 127.26.250.190:35937)
0812 03:08:59.867484 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 03:08:59.867487 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:08:59.868426 (+ 939us) server_negotiation.cc:407] Connection header received
0812 03:08:59.869887 (+ 1461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:08:59.869890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:08:59.869940 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:08:59.870027 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:08:59.871732 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:59.872300 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:59.873585 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:08:59.873816 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:08:59.873927 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:59.876906 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:08:59.876913 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:08:59.876919 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:08:59.876954 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:08:59.878911 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:59.879535 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:59.879541 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:59.879546 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:59.879653 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:08:59.880018 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:08:59.880024 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:08:59.880028 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:08:59.880321 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:08:59.880434 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 03:08:59.881016 (+ 582us) server_negotiation.cc:299] Negotiation successful
0812 03:08:59.881188 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260812 03:08:59.882228 6413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:08:59.867724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:35937 (local address 127.26.250.130:48751)
0812 03:08:59.868227 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0812 03:08:59.868286 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 03:08:59.869673 (+ 1387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:08:59.870194 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:08:59.870235 (+ 41us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:08:59.870810 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:08:59.871546 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:59.871570 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:59.872516 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:59.872519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:59.873453 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:08:59.873463 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:08:59.874015 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:08:59.874023 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:08:59.874194 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:08:59.874905 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:08:59.874933 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:08:59.876706 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:08:59.879067 (+ 2361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:59.879072 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:59.879087 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:59.879419 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:59.879764 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:08:59.879766 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:08:59.879768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:08:59.879914 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:08:59.880444 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:08:59.880449 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:08:59.880713 (+ 264us) client_negotiation.cc:769] Sending connection context
0812 03:08:59.880920 (+ 207us) client_negotiation.cc:240] Negotiation successful
0812 03:08:59.881265 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":134,"threads_started":1}
WARNING: no policy specified for kudu/127.26.250.131@KRBTEST.COM; defaulting to no policy
I20260812 03:08:59.883665 6410 heartbeater.cc:347] Connected to a master server at 127.26.250.190:35937
I20260812 03:08:59.883960 6410 heartbeater.cc:464] Registering TS with master...
I20260812 03:08:59.884601 6410 heartbeater.cc:511] Master 127.26.250.190:35937 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.131@KRBTEST.COM" created.
I20260812 03:08:59.886058 6066 ts_manager.cc:194] Registered new tserver with Master: f6da69a2f81b4dbfa59083c70cce002a (127.26.250.130:46167)
I20260812 03:08:59.886838 6066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.130@KRBTEST.COM'} at 127.26.250.130:48751
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
I20260812 03:08:59.906903 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:0
--local_ip_for_outbound_sockets=127.26.250.131
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:35937
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:44935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:09:00.031062 6416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:09:00.031415 6416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:09:00.031498 6416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:09:00.035676 6416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:09:00.035795 6416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:09:00.035893 6416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:09:00.041667 6416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:44935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:35937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:09:00.042958 6416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:09:00.044092 6416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:09:00.054797 6424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:09:00.054780 6421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:09:00.055094 6416 server_base.cc:1034] running on GCE node
W20260812 03:09:00.054780 6422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:09:00.055719 6416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:09:00.056461 6416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:09:00.057632 6416 hybrid_clock.cc:648] HybridClock initialized: now 1786504140057609 us; error 36 us; skew 500 ppm
Aug 12 03:09:00 dist-test-slave-zpfg krb5kdc[6019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504140, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:09:00.060771 6416 init.cc:377] Logged in from keytab as kudu/127.26.250.131@KRBTEST.COM (short username kudu)
I20260812 03:09:00.061815 6416 webserver.cc:460] Webserver started at http://127.26.250.131:42583/ using document root <none> and password file <none>
I20260812 03:09:00.062566 6416 fs_manager.cc:359] Metadata directory not provided
I20260812 03:09:00.062672 6416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:09:00.062981 6416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:09:00.065166 6416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/data/instance:
uuid: "e337bc8c762441fabac089617f14d33f"
format_stamp: "Formatted at 2026-08-12 03:09:00 on dist-test-slave-zpfg"
server_key: "a3e648be1729948c55f9ee5f883f6be6"
server_key_iv: "57e36bd51610cad5a2a40574c850a1d9"
server_key_version: "encryptionkey@0"
I20260812 03:09:00.065728 6416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance:
uuid: "e337bc8c762441fabac089617f14d33f"
format_stamp: "Formatted at 2026-08-12 03:09:00 on dist-test-slave-zpfg"
server_key: "a3e648be1729948c55f9ee5f883f6be6"
server_key_iv: "57e36bd51610cad5a2a40574c850a1d9"
server_key_version: "encryptionkey@0"
I20260812 03:09:00.070072 6416 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260812 03:09:00.072804 6431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:09:00.074330 6416 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 03:09:00.074523 6416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "e337bc8c762441fabac089617f14d33f"
format_stamp: "Formatted at 2026-08-12 03:09:00 on dist-test-slave-zpfg"
server_key: "a3e648be1729948c55f9ee5f883f6be6"
server_key_iv: "57e36bd51610cad5a2a40574c850a1d9"
server_key_version: "encryptionkey@0"
I20260812 03:09:00.074694 6416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:09:00.088722 6416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:09:00.092113 6416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:09:00.092360 6416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:09:00.093017 6416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:09:00.093935 6416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:09:00.094008 6416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:09:00.094084 6416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:09:00.094127 6416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:09:00.104374 6416 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:38523
I20260812 03:09:00.104413 6551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:38523 every 8 connection(s)
I20260812 03:09:00.105336 6416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:09:00.105545 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 6416
I20260812 03:09:00.105665 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance
I20260812 03:09:00.106092 27626 external_mini_cluster.cc:1428] Setting key 89cc62943d03bea67fd3c475a21541cc
Aug 12 03:09:00 dist-test-slave-zpfg krb5kdc[6019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504140, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:09:00.120981 6109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:00.107407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:47979 (local address 127.26.250.190:35937)
0812 03:09:00.107536 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 03:09:00.107562 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0812 03:09:00.108700 (+ 1138us) server_negotiation.cc:407] Connection header received
0812 03:09:00.110178 (+ 1478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:09:00.110181 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:09:00.110239 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:09:00.110321 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:09:00.111619 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.112184 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.113415 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.113653 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.113764 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:00.116783 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:09:00.116786 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:09:00.116789 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:09:00.116819 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:09:00.118404 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:09:00.118985 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:09:00.118988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:09:00.118991 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:09:00.119104 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:09:00.119549 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:09:00.119552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:09:00.119554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:09:00.119878 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:09:00.119991 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 03:09:00.120632 (+ 641us) server_negotiation.cc:299] Negotiation successful
0812 03:09:00.120782 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 03:09:00.121870 6554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:00.107927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:35937 (local address 127.26.250.131:47979)
0812 03:09:00.108508 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0812 03:09:00.108578 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 03:09:00.109949 (+ 1371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:09:00.110540 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:09:00.110547 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:09:00.110937 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:09:00.111461 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:00.111475 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.112326 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.112329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:00.113260 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:00.113278 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.113829 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.113833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:00.114016 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:00.114740 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:09:00.114759 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:09:00.116580 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:09:00.118540 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:09:00.118546 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:09:00.118560 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:09:00.118869 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:09:00.119224 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:09:00.119227 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:09:00.119231 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:09:00.119454 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:09:00.120031 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:09:00.120039 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:09:00.120362 (+ 323us) client_negotiation.cc:769] Sending connection context
0812 03:09:00.120540 (+ 178us) client_negotiation.cc:240] Negotiation successful
0812 03:09:00.120952 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":133,"threads_started":1}
I20260812 03:09:00.123124 6552 heartbeater.cc:347] Connected to a master server at 127.26.250.190:35937
I20260812 03:09:00.123385 6552 heartbeater.cc:464] Registering TS with master...
I20260812 03:09:00.123881 6552 heartbeater.cc:511] Master 127.26.250.190:35937 requested a full tablet report, sending...
I20260812 03:09:00.125126 6066 ts_manager.cc:194] Registered new tserver with Master: e337bc8c762441fabac089617f14d33f (127.26.250.131:38523)
I20260812 03:09:00.125703 6066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.131@KRBTEST.COM'} at 127.26.250.131:47979
I20260812 03:09:00.133167 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:09:00.143523 6109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:00.135514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45834 (local address 127.26.250.190:35937)
0812 03:09:00.135674 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 03:09:00.135678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:09:00.135849 (+ 171us) server_negotiation.cc:407] Connection header received
0812 03:09:00.135967 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:09:00.135970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:09:00.136019 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:09:00.136112 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:09:00.136943 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.137516 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.138418 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.138645 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.138727 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:00.140155 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:09:00.140157 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:09:00.140160 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:09:00.140184 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:09:00.141673 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:09:00.142134 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:09:00.142138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:09:00.142139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:09:00.142244 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:09:00.142568 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:09:00.142571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:09:00.142572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:09:00.142848 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:09:00.143017 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 03:09:00.143262 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 03:09:00.143377 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 03:09:00.147148 6066 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:09:00.150473 6066 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:09:00.167686 6568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:00.161502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45304 (local address 127.26.250.130:46167)
0812 03:09:00.162317 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0812 03:09:00.162353 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0812 03:09:00.162403 (+ 50us) server_negotiation.cc:407] Connection header received
0812 03:09:00.162484 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:09:00.162490 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:09:00.162685 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:09:00.162849 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:09:00.163222 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.164047 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.165856 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.166868 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.167030 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:00.167196 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 03:09:00.167379 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 03:09:00.167502 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":109,"threads_started":1}
I20260812 03:09:00.167701 6563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:00.160445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:46167 (local address 127.0.0.1:45304)
0812 03:09:00.160896 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0812 03:09:00.160951 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 03:09:00.161080 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:09:00.162994 (+ 1914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:09:00.162997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:09:00.163013 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:09:00.163088 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:00.163092 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.164228 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.164239 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:00.165712 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:00.165723 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.167036 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.167040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:00.167167 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:00.167208 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 03:09:00.167343 (+ 135us) client_negotiation.cc:240] Negotiation successful
0812 03:09:00.167502 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":78,"thread_start_us":122,"threads_started":1}
I20260812 03:09:00.167764 6565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:00.160445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:44869 (local address 127.0.0.1:36164)
0812 03:09:00.161934 (+ 1489us) negotiation.cc:107] Waiting for socket to connect
0812 03:09:00.161950 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:09:00.162053 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:09:00.162635 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:09:00.162638 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:09:00.162655 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:09:00.162785 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:00.162794 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.164424 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.164428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:00.165948 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:00.165961 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.167291 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.167298 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:00.167473 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:00.167495 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:09:00.167572 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 03:09:00.167638 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1414,"mutex_wait_us":534,"thread_start_us":94,"threads_started":1}
I20260812 03:09:00.168494 6567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:00.161347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52664 (local address 127.26.250.131:38523)
0812 03:09:00.161866 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0812 03:09:00.161887 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 03:09:00.161916 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:09:00.162009 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:09:00.162016 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:09:00.162237 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:09:00.162439 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:09:00.162811 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.164046 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.166147 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.167016 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.167144 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:00.167257 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 03:09:00.168192 (+ 935us) server_negotiation.cc:299] Negotiation successful
0812 03:09:00.168309 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":112,"threads_started":1}
I20260812 03:09:00.168819 6566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:00.160756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:38523 (local address 127.0.0.1:52664)
0812 03:09:00.161641 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0812 03:09:00.161704 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 03:09:00.161816 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:09:00.162456 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:09:00.162462 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:09:00.162490 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:09:00.162649 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:00.162670 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.164207 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.164212 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:00.165948 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:00.165961 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.167284 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.167288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:00.167429 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:00.167453 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:09:00.167525 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 03:09:00.168685 (+ 1160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"mutex_wait_us":108,"thread_start_us":110,"threads_started":1}
I20260812 03:09:00.169358 6564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:00.160641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36164 (local address 127.26.250.129:44869)
0812 03:09:00.161470 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0812 03:09:00.161478 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:09:00.162035 (+ 557us) server_negotiation.cc:407] Connection header received
0812 03:09:00.162160 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:09:00.162165 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:09:00.162363 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:09:00.162485 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:09:00.162927 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.164046 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.166141 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.166907 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.167061 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:00.167218 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 03:09:00.169035 (+ 1817us) server_negotiation.cc:299] Negotiation successful
0812 03:09:00.169173 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":706,"thread_start_us":83,"threads_started":1}
I20260812 03:09:00.171553 6338 tablet_service.cc:1467] Processing CreateTablet for tablet 0281c29f3e0a4057a6a76d10dc80a10f (DEFAULT_TABLE table=test-table [id=b7a4ff4e859747599b197e3e01af154f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:09:00.172204 6479 tablet_service.cc:1467] Processing CreateTablet for tablet 0281c29f3e0a4057a6a76d10dc80a10f (DEFAULT_TABLE table=test-table [id=b7a4ff4e859747599b197e3e01af154f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:09:00.172200 6200 tablet_service.cc:1467] Processing CreateTablet for tablet 0281c29f3e0a4057a6a76d10dc80a10f (DEFAULT_TABLE table=test-table [id=b7a4ff4e859747599b197e3e01af154f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:09:00.172904 6338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0281c29f3e0a4057a6a76d10dc80a10f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:09:00.173298 6200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0281c29f3e0a4057a6a76d10dc80a10f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:09:00.173326 6479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0281c29f3e0a4057a6a76d10dc80a10f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:09:00.180757 6569 tablet_bootstrap.cc:492] T 0281c29f3e0a4057a6a76d10dc80a10f P f6da69a2f81b4dbfa59083c70cce002a: Bootstrap starting.
I20260812 03:09:00.181804 6570 tablet_bootstrap.cc:492] T 0281c29f3e0a4057a6a76d10dc80a10f P 130ae2c6ec8c40fd81fd8ad81e63fcde: Bootstrap starting.
I20260812 03:09:00.182834 6571 tablet_bootstrap.cc:492] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f: Bootstrap starting.
I20260812 03:09:00.184361 6570 tablet_bootstrap.cc:654] T 0281c29f3e0a4057a6a76d10dc80a10f P 130ae2c6ec8c40fd81fd8ad81e63fcde: Neither blocks nor log segments found. Creating new log.
I20260812 03:09:00.184803 6569 tablet_bootstrap.cc:654] T 0281c29f3e0a4057a6a76d10dc80a10f P f6da69a2f81b4dbfa59083c70cce002a: Neither blocks nor log segments found. Creating new log.
I20260812 03:09:00.185057 6571 tablet_bootstrap.cc:654] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f: Neither blocks nor log segments found. Creating new log.
I20260812 03:09:00.185344 6570 log.cc:824] T 0281c29f3e0a4057a6a76d10dc80a10f P 130ae2c6ec8c40fd81fd8ad81e63fcde: Log is configured to *not* fsync() on all Append() calls
I20260812 03:09:00.185645 6569 log.cc:824] T 0281c29f3e0a4057a6a76d10dc80a10f P f6da69a2f81b4dbfa59083c70cce002a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:09:00.186337 6571 log.cc:824] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:09:00.187745 6570 tablet_bootstrap.cc:492] T 0281c29f3e0a4057a6a76d10dc80a10f P 130ae2c6ec8c40fd81fd8ad81e63fcde: No bootstrap required, opened a new log
I20260812 03:09:00.187976 6570 ts_tablet_manager.cc:1397] T 0281c29f3e0a4057a6a76d10dc80a10f P 130ae2c6ec8c40fd81fd8ad81e63fcde: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 03:09:00.188462 6571 tablet_bootstrap.cc:492] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f: No bootstrap required, opened a new log
I20260812 03:09:00.188663 6571 ts_tablet_manager.cc:1397] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 03:09:00.188683 6569 tablet_bootstrap.cc:492] T 0281c29f3e0a4057a6a76d10dc80a10f P f6da69a2f81b4dbfa59083c70cce002a: No bootstrap required, opened a new log
I20260812 03:09:00.188894 6569 ts_tablet_manager.cc:1397] T 0281c29f3e0a4057a6a76d10dc80a10f P f6da69a2f81b4dbfa59083c70cce002a: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260812 03:09:00.192495 6569 raft_consensus.cc:348] T 0281c29f3e0a4057a6a76d10dc80a10f P f6da69a2f81b4dbfa59083c70cce002a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130ae2c6ec8c40fd81fd8ad81e63fcde" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44869 } } peers { permanent_uuid: "f6da69a2f81b4dbfa59083c70cce002a" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46167 } } peers { permanent_uuid: "e337bc8c762441fabac089617f14d33f" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38523 } }
I20260812 03:09:00.192801 6569 raft_consensus.cc:374] T 0281c29f3e0a4057a6a76d10dc80a10f P f6da69a2f81b4dbfa59083c70cce002a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:09:00.192888 6569 raft_consensus.cc:729] T 0281c29f3e0a4057a6a76d10dc80a10f P f6da69a2f81b4dbfa59083c70cce002a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6da69a2f81b4dbfa59083c70cce002a, State: Initialized, Role: FOLLOWER
I20260812 03:09:00.192730 6571 raft_consensus.cc:348] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130ae2c6ec8c40fd81fd8ad81e63fcde" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44869 } } peers { permanent_uuid: "f6da69a2f81b4dbfa59083c70cce002a" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46167 } } peers { permanent_uuid: "e337bc8c762441fabac089617f14d33f" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38523 } }
I20260812 03:09:00.192730 6570 raft_consensus.cc:348] T 0281c29f3e0a4057a6a76d10dc80a10f P 130ae2c6ec8c40fd81fd8ad81e63fcde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130ae2c6ec8c40fd81fd8ad81e63fcde" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44869 } } peers { permanent_uuid: "f6da69a2f81b4dbfa59083c70cce002a" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46167 } } peers { permanent_uuid: "e337bc8c762441fabac089617f14d33f" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38523 } }
I20260812 03:09:00.193007 6570 raft_consensus.cc:374] T 0281c29f3e0a4057a6a76d10dc80a10f P 130ae2c6ec8c40fd81fd8ad81e63fcde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:09:00.193058 6571 raft_consensus.cc:374] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:09:00.193094 6570 raft_consensus.cc:729] T 0281c29f3e0a4057a6a76d10dc80a10f P 130ae2c6ec8c40fd81fd8ad81e63fcde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130ae2c6ec8c40fd81fd8ad81e63fcde, State: Initialized, Role: FOLLOWER
I20260812 03:09:00.193120 6571 raft_consensus.cc:729] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e337bc8c762441fabac089617f14d33f, State: Initialized, Role: FOLLOWER
I20260812 03:09:00.193478 6569 consensus_queue.cc:260] T 0281c29f3e0a4057a6a76d10dc80a10f P f6da69a2f81b4dbfa59083c70cce002a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130ae2c6ec8c40fd81fd8ad81e63fcde" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44869 } } peers { permanent_uuid: "f6da69a2f81b4dbfa59083c70cce002a" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46167 } } peers { permanent_uuid: "e337bc8c762441fabac089617f14d33f" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38523 } }
I20260812 03:09:00.193612 6571 consensus_queue.cc:260] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130ae2c6ec8c40fd81fd8ad81e63fcde" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44869 } } peers { permanent_uuid: "f6da69a2f81b4dbfa59083c70cce002a" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46167 } } peers { permanent_uuid: "e337bc8c762441fabac089617f14d33f" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38523 } }
I20260812 03:09:00.193612 6570 consensus_queue.cc:260] T 0281c29f3e0a4057a6a76d10dc80a10f P 130ae2c6ec8c40fd81fd8ad81e63fcde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130ae2c6ec8c40fd81fd8ad81e63fcde" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44869 } } peers { permanent_uuid: "f6da69a2f81b4dbfa59083c70cce002a" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46167 } } peers { permanent_uuid: "e337bc8c762441fabac089617f14d33f" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38523 } }
I20260812 03:09:00.194373 6410 heartbeater.cc:503] Master 127.26.250.190:35937 was elected leader, sending a full tablet report...
I20260812 03:09:00.194702 6570 ts_tablet_manager.cc:1428] T 0281c29f3e0a4057a6a76d10dc80a10f P 130ae2c6ec8c40fd81fd8ad81e63fcde: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 03:09:00.194702 6571 ts_tablet_manager.cc:1428] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260812 03:09:00.194707 6569 ts_tablet_manager.cc:1428] T 0281c29f3e0a4057a6a76d10dc80a10f P f6da69a2f81b4dbfa59083c70cce002a: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 03:09:00.195060 6552 heartbeater.cc:503] Master 127.26.250.190:35937 was elected leader, sending a full tablet report...
I20260812 03:09:00.195066 6267 heartbeater.cc:503] Master 127.26.250.190:35937 was elected leader, sending a full tablet report...
I20260812 03:09:00.211084 6577 raft_consensus.cc:482] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:09:00.211391 6577 raft_consensus.cc:504] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130ae2c6ec8c40fd81fd8ad81e63fcde" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44869 } } peers { permanent_uuid: "f6da69a2f81b4dbfa59083c70cce002a" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46167 } } peers { permanent_uuid: "e337bc8c762441fabac089617f14d33f" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38523 } }
I20260812 03:09:00.212623 6577 leader_election.cc:288] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 130ae2c6ec8c40fd81fd8ad81e63fcde (127.26.250.129:44869), f6da69a2f81b4dbfa59083c70cce002a (127.26.250.130:46167)
I20260812 03:09:00.217185 6554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:00.212966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:46167 (local address 127.26.250.131:56211)
0812 03:09:00.213123 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 03:09:00.213141 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:09:00.213254 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:09:00.213641 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:09:00.213644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:09:00.213664 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:09:00.213798 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:00.213856 (+ 58us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.214901 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.214904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:00.215929 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:00.215939 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.216669 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.216673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:00.216797 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:00.216862 (+ 65us) client_negotiation.cc:769] Sending connection context
0812 03:09:00.216943 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 03:09:00.217035 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 03:09:00.217203 6568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:00.213242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:56211 (local address 127.26.250.130:46167)
0812 03:09:00.213425 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 03:09:00.213429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:09:00.213441 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:09:00.213488 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:09:00.213491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:09:00.213531 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:09:00.213624 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:09:00.213999 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.214774 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.216072 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.216529 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.216628 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:00.216670 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 03:09:00.216977 (+ 307us) server_negotiation.cc:299] Negotiation successful
0812 03:09:00.217071 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260812 03:09:00.217255 6578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:00.212973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:44869 (local address 127.26.250.131:52943)
0812 03:09:00.213709 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0812 03:09:00.213721 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:09:00.213825 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:09:00.214100 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:09:00.214103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:09:00.214121 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:09:00.214242 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:00.214248 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.215112 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.215114 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:00.215975 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:00.215985 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.216830 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.216833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:00.216941 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:00.216967 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 03:09:00.217045 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 03:09:00.217110 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"mutex_wait_us":343,"thread_start_us":156,"threads_started":1}
I20260812 03:09:00.217293 6564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:00.213116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:52943 (local address 127.26.250.129:44869)
0812 03:09:00.213288 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 03:09:00.213294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:09:00.213820 (+ 526us) server_negotiation.cc:407] Connection header received
0812 03:09:00.213936 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:09:00.213940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:09:00.214003 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:09:00.214061 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:09:00.214453 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.214993 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.216158 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.216697 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.216814 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:00.216864 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 03:09:00.217077 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 03:09:00.217160 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 03:09:00.217932 6220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0281c29f3e0a4057a6a76d10dc80a10f" candidate_uuid: "e337bc8c762441fabac089617f14d33f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130ae2c6ec8c40fd81fd8ad81e63fcde" is_pre_election: true
I20260812 03:09:00.217928 6361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0281c29f3e0a4057a6a76d10dc80a10f" candidate_uuid: "e337bc8c762441fabac089617f14d33f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6da69a2f81b4dbfa59083c70cce002a" is_pre_election: true
I20260812 03:09:00.218264 6220 raft_consensus.cc:2454] T 0281c29f3e0a4057a6a76d10dc80a10f P 130ae2c6ec8c40fd81fd8ad81e63fcde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e337bc8c762441fabac089617f14d33f in term 0.
I20260812 03:09:00.218271 6361 raft_consensus.cc:2454] T 0281c29f3e0a4057a6a76d10dc80a10f P f6da69a2f81b4dbfa59083c70cce002a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e337bc8c762441fabac089617f14d33f in term 0.
I20260812 03:09:00.218828 6434 leader_election.cc:302] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130ae2c6ec8c40fd81fd8ad81e63fcde, e337bc8c762441fabac089617f14d33f; no voters:
I20260812 03:09:00.219128 6577 raft_consensus.cc:2781] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:09:00.219231 6577 raft_consensus.cc:482] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:09:00.219321 6577 raft_consensus.cc:3037] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:09:00.220465 6577 raft_consensus.cc:504] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130ae2c6ec8c40fd81fd8ad81e63fcde" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44869 } } peers { permanent_uuid: "f6da69a2f81b4dbfa59083c70cce002a" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46167 } } peers { permanent_uuid: "e337bc8c762441fabac089617f14d33f" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38523 } }
I20260812 03:09:00.220856 6577 leader_election.cc:288] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f [CANDIDATE]: Term 1 election: Requested vote from peers 130ae2c6ec8c40fd81fd8ad81e63fcde (127.26.250.129:44869), f6da69a2f81b4dbfa59083c70cce002a (127.26.250.130:46167)
I20260812 03:09:00.221364 6220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0281c29f3e0a4057a6a76d10dc80a10f" candidate_uuid: "e337bc8c762441fabac089617f14d33f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130ae2c6ec8c40fd81fd8ad81e63fcde"
I20260812 03:09:00.221500 6220 raft_consensus.cc:3037] T 0281c29f3e0a4057a6a76d10dc80a10f P 130ae2c6ec8c40fd81fd8ad81e63fcde [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:09:00.222548 6361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0281c29f3e0a4057a6a76d10dc80a10f" candidate_uuid: "e337bc8c762441fabac089617f14d33f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6da69a2f81b4dbfa59083c70cce002a"
I20260812 03:09:00.222709 6361 raft_consensus.cc:3037] T 0281c29f3e0a4057a6a76d10dc80a10f P f6da69a2f81b4dbfa59083c70cce002a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:09:00.222744 6220 raft_consensus.cc:2454] T 0281c29f3e0a4057a6a76d10dc80a10f P 130ae2c6ec8c40fd81fd8ad81e63fcde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e337bc8c762441fabac089617f14d33f in term 1.
I20260812 03:09:00.223088 6434 leader_election.cc:302] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130ae2c6ec8c40fd81fd8ad81e63fcde, e337bc8c762441fabac089617f14d33f; no voters:
I20260812 03:09:00.223306 6577 raft_consensus.cc:2781] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:09:00.223611 6577 raft_consensus.cc:686] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f [term 1 LEADER]: Becoming Leader. State: Replica: e337bc8c762441fabac089617f14d33f, State: Running, Role: LEADER
I20260812 03:09:00.224061 6577 consensus_queue.cc:237] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130ae2c6ec8c40fd81fd8ad81e63fcde" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44869 } } peers { permanent_uuid: "f6da69a2f81b4dbfa59083c70cce002a" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46167 } } peers { permanent_uuid: "e337bc8c762441fabac089617f14d33f" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38523 } }
I20260812 03:09:00.224197 6361 raft_consensus.cc:2454] T 0281c29f3e0a4057a6a76d10dc80a10f P f6da69a2f81b4dbfa59083c70cce002a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e337bc8c762441fabac089617f14d33f in term 1.
I20260812 03:09:00.227838 6063 catalog_manager.cc:5495] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f reported cstate change: term changed from 0 to 1, leader changed from <none> to e337bc8c762441fabac089617f14d33f (127.26.250.131). New cstate: current_term: 1 leader_uuid: "e337bc8c762441fabac089617f14d33f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "130ae2c6ec8c40fd81fd8ad81e63fcde" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6da69a2f81b4dbfa59083c70cce002a" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e337bc8c762441fabac089617f14d33f" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38523 } health_report { overall_health: HEALTHY } } }
I20260812 03:09:00.249986 6567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:00.245378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52674 (local address 127.26.250.131:38523)
0812 03:09:00.245528 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 03:09:00.245533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:09:00.245549 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:09:00.245720 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:09:00.245723 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:09:00.245789 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:09:00.245922 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:09:00.246204 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.247123 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.248337 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:00.248590 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:00.248711 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:00.249091 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:09:00.249562 (+ 471us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:09:00.249665 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 03:09:00.249745 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 03:09:00.249841 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:09:00.256523 6361 raft_consensus.cc:1261] T 0281c29f3e0a4057a6a76d10dc80a10f P f6da69a2f81b4dbfa59083c70cce002a [term 1 FOLLOWER]: Refusing update from remote peer e337bc8c762441fabac089617f14d33f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:09:00.256557 6220 raft_consensus.cc:1261] T 0281c29f3e0a4057a6a76d10dc80a10f P 130ae2c6ec8c40fd81fd8ad81e63fcde [term 1 FOLLOWER]: Refusing update from remote peer e337bc8c762441fabac089617f14d33f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:09:00.257364 6577 consensus_queue.cc:1035] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6da69a2f81b4dbfa59083c70cce002a" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:09:00.257575 6579 consensus_queue.cc:1035] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f [LEADER]: Connected to new peer: Peer: permanent_uuid: "130ae2c6ec8c40fd81fd8ad81e63fcde" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 44869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:09:00.266567 6583 mvcc.cc:205] Tried to move back new op lower bound from 7317520958482927616 to 7317520958361436160. Current Snapshot: MvccSnapshot[applied={T|T < 7317520958482927616}]
I20260812 03:09:00.274610 6066 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:09:00.274848 6066 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:09:00.277726 6066 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2ac4e629553443cfaa1984eaa8ed8228: Sending DeleteTablet for 3 replicas of tablet 0281c29f3e0a4057a6a76d10dc80a10f
I20260812 03:09:00.278795 6338 tablet_service.cc:1514] Processing DeleteTablet for tablet 0281c29f3e0a4057a6a76d10dc80a10f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:09:00 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:45304
I20260812 03:09:00.278785 6479 tablet_service.cc:1514] Processing DeleteTablet for tablet 0281c29f3e0a4057a6a76d10dc80a10f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:09:00 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:52664
I20260812 03:09:00.278868 6200 tablet_service.cc:1514] Processing DeleteTablet for tablet 0281c29f3e0a4057a6a76d10dc80a10f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:09:00 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:36164
I20260812 03:09:00.279556 6593 tablet_replica.cc:330] T 0281c29f3e0a4057a6a76d10dc80a10f P f6da69a2f81b4dbfa59083c70cce002a: stopping tablet replica
I20260812 03:09:00.279734 6592 tablet_replica.cc:330] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f: stopping tablet replica
I20260812 03:09:00.280005 6593 raft_consensus.cc:2229] T 0281c29f3e0a4057a6a76d10dc80a10f P f6da69a2f81b4dbfa59083c70cce002a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:09:00.279475 6594 tablet_replica.cc:330] T 0281c29f3e0a4057a6a76d10dc80a10f P 130ae2c6ec8c40fd81fd8ad81e63fcde: stopping tablet replica
I20260812 03:09:00.280120 6592 raft_consensus.cc:2229] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:09:00.280467 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 6132
I20260812 03:09:00.280485 6593 raft_consensus.cc:2258] T 0281c29f3e0a4057a6a76d10dc80a10f P f6da69a2f81b4dbfa59083c70cce002a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:09:00.280512 6594 raft_consensus.cc:2229] T 0281c29f3e0a4057a6a76d10dc80a10f P 130ae2c6ec8c40fd81fd8ad81e63fcde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:09:00.280699 6592 raft_consensus.cc:2258] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:09:00.282582 6592 ts_tablet_manager.cc:1905] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:09:00.286111 6592 ts_tablet_manager.cc:1918] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:09:00.286289 6592 log.cc:1196] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/0281c29f3e0a4057a6a76d10dc80a10f
I20260812 03:09:00.286522 6593 ts_tablet_manager.cc:1905] T 0281c29f3e0a4057a6a76d10dc80a10f P f6da69a2f81b4dbfa59083c70cce002a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:09:00.286792 6592 ts_tablet_manager.cc:1939] T 0281c29f3e0a4057a6a76d10dc80a10f P e337bc8c762441fabac089617f14d33f: Deleting consensus metadata
I20260812 03:09:00.288443 6053 catalog_manager.cc:4841] TS e337bc8c762441fabac089617f14d33f (127.26.250.131:38523): tablet 0281c29f3e0a4057a6a76d10dc80a10f (table test-table [id=b7a4ff4e859747599b197e3e01af154f]) successfully deleted
I20260812 03:09:00.289911 6593 ts_tablet_manager.cc:1918] T 0281c29f3e0a4057a6a76d10dc80a10f P f6da69a2f81b4dbfa59083c70cce002a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:09:00.290042 6593 log.cc:1196] T 0281c29f3e0a4057a6a76d10dc80a10f P f6da69a2f81b4dbfa59083c70cce002a: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786504075277362-27626-0/minicluster-data/ts-1/wal/wals/0281c29f3e0a4057a6a76d10dc80a10f
I20260812 03:09:00.290531 6593 ts_tablet_manager.cc:1939] T 0281c29f3e0a4057a6a76d10dc80a10f P f6da69a2f81b4dbfa59083c70cce002a: Deleting consensus metadata
I20260812 03:09:00.292161 6055 catalog_manager.cc:4841] TS f6da69a2f81b4dbfa59083c70cce002a (127.26.250.130:46167): tablet 0281c29f3e0a4057a6a76d10dc80a10f (table test-table [id=b7a4ff4e859747599b197e3e01af154f]) successfully deleted
W20260812 03:09:00.294202 6054 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.250.129:44869 (error 108)
I20260812 03:09:00.295221 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 6272
I20260812 03:09:00.295495 6566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:00.295132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:44869 (local address 127.0.0.1:36166)
0812 03:09:00.295303 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 03:09:00.295385 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.129:44869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260812 03:09:00.301137 6054 catalog_manager.cc:4584] TS 130ae2c6ec8c40fd81fd8ad81e63fcde (127.26.250.129:44869): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0281c29f3e0a4057a6a76d10dc80a10f: Network error: Client connection negotiation failed: client connection to 127.26.250.129:44869: connect: Connection refused (error 111)
I20260812 03:09:00.309311 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 6416
I20260812 03:09:00.318802 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 6033
2026-08-12T03:09:00Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2477 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:09:00 dist-test-slave-zpfg krb5kdc[6598](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:09:00 dist-test-slave-zpfg krb5kdc[6598](info): set up 2 sockets
Aug 12 03:09:00 dist-test-slave-zpfg krb5kdc[6598](info): commencing operation
krb5kdc: starting...
W20260812 03:09:01.468760 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:09:01 dist-test-slave-zpfg krb5kdc[6598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:09:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:09:01Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.26.250.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.26.250.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.26.250.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/rangeradmin_127.26.250.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.26.250.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/rangeradmin_127.26.250.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.26.250.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.26.250.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.26.250.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/rangerlookup_127.26.250.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.26.250.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/rangerlookup_127.26.250.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.250.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.250.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.250.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/HTTP_127.26.250.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.250.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/HTTP_127.26.250.148@KRBTEST.COM.keytab.
I20260812 03:09:01.673936 27626 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskh6ikwC/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/postgres -l logfile start
2026-08-12 03:09:04.678 UTC [6630] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 03:09:04.678 UTC [6630] LOG: listening on IPv4 address "127.26.250.148", port 40329
2026-08-12 03:09:04.683 UTC [6630] LOG: listening on Unix socket "/tmp/.s.PGSQL.40329"
2026-08-12 03:09:04.707 UTC [6634] LOG: database system was shut down at 2026-08-12 03:09:02 UTC
2026-08-12 03:09:04.711 UTC [6630] LOG: database system is ready to accept connections
I20260812 03:09:05.721437 27626 mini_postgres.cc:96] Postgres bound to 40329
2026-08-12 03:09:05.730 UTC [6642] FATAL: database "slave" does not exist
127.26.250.148:40329 - accepting connections
I20260812 03:09:05.731393 27626 mini_ranger.cc:161] Starting Ranger...
I20260812 03:09:05.756188 27626 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 03:09:06.361547 27626 mini_ranger.cc:87] Created ranger Postgres database
I20260812 03:09:06.361663 27626 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ranger-admin
2026-08-12 03:09:06,824 [I] DB FLAVOR :POSTGRES
2026-08-12 03:09:06,825 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 03:09:06,825 [I] Checking connection..
2026-08-12 03:09:06,825 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:40329/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 03:09:07,096 [I] Checking connection passed.
2026-08-12 03:09:07,097 [I] --------- Verifying version history table ---------
2026-08-12 03:09:07,097 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:40329/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:09:07,394 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 03:09:07,394 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:40329/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:09:07,642 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 03:09:07,643 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 03:09:07,643 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:40329/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 03:09:07.892 UTC [6708] WARNING: there is no transaction in progress
2026-08-12 03:09:07,900 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 03:09:07,901 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:40329/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:09:08,187 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 03:09:08,187 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 03:09:08,187 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:40329/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 03:09:08,466 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:40329/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 03:09:08,732 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:40329/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zpfg.c.gcp-upstream.internal','N') ;"
2026-08-12 03:09:08,998 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:40329/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 03:09:09,291 [I] Table x_portal_user does not exist in database ranger
2026-08-12 03:09:09,291 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:40329/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 03:09:09,559 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 03:09:09,559 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:40329/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 03:09:09,806 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:40329/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 03:09:10,087 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 03:09:10,088 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:40329/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 03:09:11.266 UTC [6831] WARNING: there is no transaction in progress
2026-08-12 03:09:11.288 UTC [6831] WARNING: there is no transaction in progress
2026-08-12 03:09:11.312 UTC [6831] WARNING: there is no transaction in progress
2026-08-12 03:09:11.332 UTC [6831] WARNING: there is no transaction in progress
2026-08-12 03:09:11.355 UTC [6831] WARNING: there is no transaction in progress
2026-08-12 03:09:11.377 UTC [6831] WARNING: there is no transaction in progress
2026-08-12 03:09:11.447 UTC [6831] WARNING: there is no transaction in progress
2026-08-12 03:09:11.456 UTC [6831] WARNING: there is no transaction in progress
2026-08-12 03:09:11.474 UTC [6831] WARNING: there is no transaction in progress
2026-08-12 03:09:11.486 UTC [6831] WARNING: there is no transaction in progress
2026-08-12 03:09:11.498 UTC [6831] WARNING: there is no transaction in progress
2026-08-12 03:09:11.515 UTC [6831] WARNING: there is no transaction in progress
2026-08-12 03:09:11.527 UTC [6831] WARNING: there is no transaction in progress
2026-08-12 03:09:11.535 UTC [6831] WARNING: there is no transaction in progress
2026-08-12 03:09:11.544 UTC [6831] WARNING: there is no transaction in progress
2026-08-12 03:09:11.552 UTC [6831] WARNING: there is no transaction in progress
2026-08-12 03:09:12,448 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 03:09:12,448 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:40329/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 03:09:12,711 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 03:09:12,712 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:40329/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 03:09:12,975 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 03:09:12,975 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:40329/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 03:09:13,236 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 03:09:13,236 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:40329/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 03:09:13,509 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:40329/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 03:09:13,797 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:40329/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zpfg.c.gcp-upstream.internal';"
2026-08-12 03:09:14,070 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 03:09:14,070 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:40329/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 03:09:14,345 [I] DB_PATCHES have already been applied
I20260812 03:09:14.352522 27626 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ranger-admin:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskh6ikwC/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 03:09:14.352604 27626 mini_ranger.cc:193] Using host: 127.26.250.148
I20260812 03:09:14.357892 27626 mini_ranger.cc:227] Ranger admin URL: http://127.26.250.148:43819
Aug 12, 2026 3:09:14 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 3:09:14 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 3:09:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 3:09:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 3:09:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 3:09:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 3:09:15 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 03:09:15.603886 27626 mini_ranger.cc:160] Time spent starting Ranger: real 9.873s user 0.000s sys 0.015s
Aug 12, 2026 3:09:37 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 3:09:37 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 3:09:37 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 3:09:37 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 3:09:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 3:09:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 3:09:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 3:09:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 3:09:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 3:09:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 3:09:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 3:09:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 3:09:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 3:09:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 3:09:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 3:09:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 3:09:37 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 3:09:38 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 03:09:39.463927 27626 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.26.250.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.26.250.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.26.250.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/rangerkms_127.26.250.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.26.250.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/rangerkms_127.26.250.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.250.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.250.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.250.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/HTTP_127.26.250.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.250.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/HTTP_127.26.250.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 03:09:39 dist-test-slave-zpfg krb5kdc[6598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504179, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 03:09:39.564409 27626 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 03:09:39.576434 27626 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 03:09:40.655604 27626 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 03:09:40.655718 27626 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ranger-kms
I20260812 03:09:40.655892 27626 mini_ranger_kms.cc:227] Using postgres at 127.26.250.148:40329
2026-08-12 03:09:41,121 [I] DB FLAVOR :POSTGRES
2026-08-12 03:09:41,122 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 03:09:41,122 [I] Checking connection
2026-08-12 03:09:41,122 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:40329/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 03:09:41,429 [I] connection success
2026-08-12 03:09:41,429 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 03:09:41,429 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 03:09:41,429 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:40329/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 03:09:41,725 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 03:09:41,725 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 03:09:41,725 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 03:09:41,726 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:40329/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 03:09:42,073 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 03:09:42.647972 27626 mini_ranger_kms.cc:310] Created kms service
I20260812 03:09:42.899199 27626 mini_ranger_kms.cc:326] Created kudu user
I20260812 03:09:42.951287 27626 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 03:09:42 dist-test-slave-zpfg krb5kdc[6598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504179, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.26.250.148@KRBTEST.COM
I20260812 03:09:43.413362 27626 mini_ranger_kms.cc:384] Added ranger policy
I20260812 03:09:43.413734 27626 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ranger-kms:/tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 03:09:43.413843 27626 mini_ranger_kms.cc:240] Using host: 127.26.250.148
I20260812 03:09:43.420188 27626 mini_ranger_kms.cc:276] Ranger KMS PID: 7030
I20260812 03:09:43.420372 27626 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.26.250.148:50291
Aug 12, 2026 3:09:44 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 3:09:44 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 3:09:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 3:09:44 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 3:09:45 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 3:09:45 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260812 03:09:45.820415 27626 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.256s user 0.003s sys 0.433s
I20260812 03:09:45.820585 27626 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 03:09:45.820731 27626 mini_ranger_kms.cc:401] 127.26.250.148:50291/kms/v1/keys
Aug 12 03:09:46 dist-test-slave-zpfg krb5kdc[6598](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786504186, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.250.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 03:09:46 dist-test-slave-zpfg krb5kdc[6598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504186, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.250.148@KRBTEST.COM for HTTP/127.26.250.148@KRBTEST.COM
Aug 12 03:09:47 dist-test-slave-zpfg krb5kdc[6598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504186, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.250.148@KRBTEST.COM for HTTP/127.26.250.148@KRBTEST.COM
Aug 12 03:09:49 dist-test-slave-zpfg krb5kdc[6598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:09:49.865769 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:35521
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:40681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:35521
--ranger_config_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.250.148:50291/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:09:50.004911 7081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:09:50.005239 7081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:09:50.005283 7081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:09:50.009992 7081 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:09:50.010099 7081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:09:50.010120 7081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:09:50.010138 7081 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:09:50.010154 7081 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:09:50.016654 7081 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:40681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/master-0/wal
--ranger_kms_url=127.26.250.148:50291/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:35521
--ranger_config_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:35521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:09:50.017982 7081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:09:50.019270 7081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:09:50.030735 7086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:09:50.030943 7089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:09:50.030799 7081 server_base.cc:1034] running on GCE node
W20260812 03:09:50.031095 7087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:09:50.031922 7081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:09:50.033969 7081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:09:50.035216 7081 hybrid_clock.cc:648] HybridClock initialized: now 1786504190035168 us; error 60 us; skew 500 ppm
Aug 12 03:09:50 dist-test-slave-zpfg krb5kdc[6598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504190, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:09:50.039543 7081 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:09:50.040695 7081 webserver.cc:460] Webserver started at http://127.26.250.190:40909/ using document root <none> and password file <none>
I20260812 03:09:50.041301 7081 fs_manager.cc:359] Metadata directory not provided
I20260812 03:09:50.041390 7081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:09:50.041637 7081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:09:50 dist-test-slave-zpfg krb5kdc[6598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504190, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for HTTP/127.26.250.148@KRBTEST.COM
I20260812 03:09:50.099036 7081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "0d6610f5a38442c99555510cba878ef0"
format_stamp: "Formatted at 2026-08-12 03:09:50 on dist-test-slave-zpfg"
server_key: "442b6376a277981e3c852b33a9041120"
server_key_iv: "5ce6f13d4650d2b2faf7b8c1a181e964"
server_key_version: "kuduclusterkey@0"
I20260812 03:09:50.099777 7081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "0d6610f5a38442c99555510cba878ef0"
format_stamp: "Formatted at 2026-08-12 03:09:50 on dist-test-slave-zpfg"
server_key: "442b6376a277981e3c852b33a9041120"
server_key_iv: "5ce6f13d4650d2b2faf7b8c1a181e964"
server_key_version: "kuduclusterkey@0"
I20260812 03:09:50.104427 7081 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:09:50.121193 7096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:09:50.122865 7081 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:09:50.123051 7081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "0d6610f5a38442c99555510cba878ef0"
format_stamp: "Formatted at 2026-08-12 03:09:50 on dist-test-slave-zpfg"
server_key: "442b6376a277981e3c852b33a9041120"
server_key_iv: "5ce6f13d4650d2b2faf7b8c1a181e964"
server_key_version: "kuduclusterkey@0"
I20260812 03:09:50.123210 7081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:09:50.146111 7081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:09:50.150018 7081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:09:50.150247 7081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:09:50.159353 7081 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:35521
I20260812 03:09:50.159401 7153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:35521 every 8 connection(s)
I20260812 03:09:50.160494 7081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:09:50.163645 7154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:09:50.167193 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 7081
I20260812 03:09:50.167361 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:09:50.167749 27626 external_mini_cluster.cc:1428] Setting key 6e01495c885db23416af0119832e3b0a
I20260812 03:09:50.172139 7154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d6610f5a38442c99555510cba878ef0: Bootstrap starting.
I20260812 03:09:50.175633 7154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d6610f5a38442c99555510cba878ef0: Neither blocks nor log segments found. Creating new log.
I20260812 03:09:50.176764 7154 log.cc:824] T 00000000000000000000000000000000 P 0d6610f5a38442c99555510cba878ef0: Log is configured to *not* fsync() on all Append() calls
I20260812 03:09:50.179589 7154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d6610f5a38442c99555510cba878ef0: No bootstrap required, opened a new log
Aug 12 03:09:50 dist-test-slave-zpfg krb5kdc[6598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:09:50.183988 7154 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d6610f5a38442c99555510cba878ef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d6610f5a38442c99555510cba878ef0" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 35521 } }
I20260812 03:09:50.184333 7154 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d6610f5a38442c99555510cba878ef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:09:50.184429 7154 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d6610f5a38442c99555510cba878ef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d6610f5a38442c99555510cba878ef0, State: Initialized, Role: FOLLOWER
I20260812 03:09:50.185015 7154 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d6610f5a38442c99555510cba878ef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d6610f5a38442c99555510cba878ef0" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 35521 } }
I20260812 03:09:50.185187 7154 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d6610f5a38442c99555510cba878ef0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:09:50.185285 7154 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d6610f5a38442c99555510cba878ef0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:09:50.185415 7154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d6610f5a38442c99555510cba878ef0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:09:50.186787 7154 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d6610f5a38442c99555510cba878ef0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d6610f5a38442c99555510cba878ef0" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 35521 } }
I20260812 03:09:50.187211 7154 leader_election.cc:302] T 00000000000000000000000000000000 P 0d6610f5a38442c99555510cba878ef0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d6610f5a38442c99555510cba878ef0; no voters:
I20260812 03:09:50.187597 7157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:50.169635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47756 (local address 127.26.250.190:35521)
0812 03:09:50.170213 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0812 03:09:50.170235 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 03:09:50.170287 (+ 52us) server_negotiation.cc:407] Connection header received
0812 03:09:50.171226 (+ 939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:09:50.171252 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:09:50.171754 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:09:50.172157 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:09:50.173215 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:50.174853 (+ 1638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:50.176384 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:50.177258 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:50.177482 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:50.181264 (+ 3782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:09:50.181274 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:09:50.181292 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:09:50.181328 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:09:50.183340 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:09:50.183956 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:09:50.183962 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:09:50.183967 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:09:50.184149 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:09:50.184524 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:09:50.184528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:09:50.184530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:09:50.185090 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:09:50.186072 (+ 982us) server_negotiation.cc:1035] Waiting for connection context
0812 03:09:50.186219 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 03:09:50.186563 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":174,"threads_started":1}
I20260812 03:09:50.187611 7154 leader_election.cc:288] T 00000000000000000000000000000000 P 0d6610f5a38442c99555510cba878ef0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:09:50.187970 7159 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d6610f5a38442c99555510cba878ef0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:09:50.188289 7159 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d6610f5a38442c99555510cba878ef0 [term 1 LEADER]: Becoming Leader. State: Replica: 0d6610f5a38442c99555510cba878ef0, State: Running, Role: LEADER
I20260812 03:09:50.188715 7159 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d6610f5a38442c99555510cba878ef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d6610f5a38442c99555510cba878ef0" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 35521 } }
I20260812 03:09:50.189519 7154 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d6610f5a38442c99555510cba878ef0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:09:50.191860 7162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d6610f5a38442c99555510cba878ef0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d6610f5a38442c99555510cba878ef0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d6610f5a38442c99555510cba878ef0" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 35521 } } }
I20260812 03:09:50.192027 7162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d6610f5a38442c99555510cba878ef0 [sys.catalog]: This master's current role is: LEADER
I20260812 03:09:50.192233 7154 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 03:09:50.192363 7161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d6610f5a38442c99555510cba878ef0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d6610f5a38442c99555510cba878ef0. Latest consensus state: current_term: 1 leader_uuid: "0d6610f5a38442c99555510cba878ef0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d6610f5a38442c99555510cba878ef0" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 35521 } } }
I20260812 03:09:50.192503 7161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d6610f5a38442c99555510cba878ef0 [sys.catalog]: This master's current role is: LEADER
I20260812 03:09:50.193660 7174 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:09:50.201246 7174 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:09:50.208794 7174 catalog_manager.cc:1347] Generated new cluster ID: 31524cf9c4fd4650aef31aa31cdcd6ab
I20260812 03:09:50.208876 7174 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:09:50.241076 7174 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:09:50.242738 7174 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:09:50.257225 7174 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d6610f5a38442c99555510cba878ef0: Generated new TSK 0
I20260812 03:09:50.258458 7174 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.250.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
I20260812 03:09:51.162930 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.250.148:50291/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:0
--local_ip_for_outbound_sockets=127.26.250.129
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:35521
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:40681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:09:51.352110 7202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:09:51.352613 7202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:09:51.352743 7202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:09:51.358901 7202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:09:51.359089 7202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:09:51.359251 7202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:09:51.368531 7202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:40681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.26.250.148:50291/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:35521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:09:51.370468 7202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:09:51.372011 7202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:09:51.386214 7209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:09:51.386608 7202 server_base.cc:1034] running on GCE node
W20260812 03:09:51.386849 7212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:09:51.386263 7210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:09:51.387583 7202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:09:51.388520 7202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:09:51.389748 7202 hybrid_clock.cc:648] HybridClock initialized: now 1786504191389704 us; error 61 us; skew 500 ppm
Aug 12 03:09:51 dist-test-slave-zpfg krb5kdc[6598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504191, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:09:51.394435 7202 init.cc:377] Logged in from keytab as kudu/127.26.250.129@KRBTEST.COM (short username kudu)
I20260812 03:09:51.395829 7202 webserver.cc:460] Webserver started at http://127.26.250.129:46355/ using document root <none> and password file <none>
I20260812 03:09:51.396744 7202 fs_manager.cc:359] Metadata directory not provided
I20260812 03:09:51.396958 7202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:09:51.397316 7202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:09:51 dist-test-slave-zpfg krb5kdc[6598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504191, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for HTTP/127.26.250.148@KRBTEST.COM
I20260812 03:09:51.425040 7202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-0/data/instance:
uuid: "1933f9c0ddd6409ea0855754142f3068"
format_stamp: "Formatted at 2026-08-12 03:09:51 on dist-test-slave-zpfg"
server_key: "f732ec220ce1884458a6978a4c180efa"
server_key_iv: "23610c0a787654912c4def70933a193b"
server_key_version: "kuduclusterkey@0"
I20260812 03:09:51.426002 7202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance:
uuid: "1933f9c0ddd6409ea0855754142f3068"
format_stamp: "Formatted at 2026-08-12 03:09:51 on dist-test-slave-zpfg"
server_key: "f732ec220ce1884458a6978a4c180efa"
server_key_iv: "23610c0a787654912c4def70933a193b"
server_key_version: "kuduclusterkey@0"
I20260812 03:09:51.433413 7202 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.000s
I20260812 03:09:51.466964 7221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:09:51.471452 7202 fs_manager.cc:725] Time spent opening block manager: real 0.019s user 0.003s sys 0.001s
I20260812 03:09:51.471657 7202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "1933f9c0ddd6409ea0855754142f3068"
format_stamp: "Formatted at 2026-08-12 03:09:51 on dist-test-slave-zpfg"
server_key: "f732ec220ce1884458a6978a4c180efa"
server_key_iv: "23610c0a787654912c4def70933a193b"
server_key_version: "kuduclusterkey@0"
I20260812 03:09:51.471832 7202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:09:51.509534 7202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:09:51.513989 7202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:09:51.514235 7202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:09:51.514966 7202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:09:51.515974 7202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:09:51.516048 7202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:09:51.516172 7202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:09:51.516232 7202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:09:51.531342 7202 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:34759
I20260812 03:09:51.531468 7339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:34759 every 8 connection(s)
I20260812 03:09:51.532598 7202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:09:51.537811 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 7202
I20260812 03:09:51.537961 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance
I20260812 03:09:51.538287 27626 external_mini_cluster.cc:1428] Setting key dd18c60826cba26e728cbda0663224d0
Aug 12 03:09:51 dist-test-slave-zpfg krb5kdc[6598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504191, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:09:51.553202 7343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:51.536609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:52467 (local address 127.26.250.190:35521)
0812 03:09:51.536918 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0812 03:09:51.536924 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:09:51.536945 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:09:51.538957 (+ 2012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:09:51.538960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:09:51.539019 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:09:51.539111 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:09:51.541091 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:51.542037 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:51.543608 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:51.543858 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:51.543983 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:51.547603 (+ 3620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:09:51.547606 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:09:51.547608 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:09:51.547635 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:09:51.549898 (+ 2263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:09:51.550843 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:09:51.550849 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:09:51.550851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:09:51.550976 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:09:51.551499 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:09:51.551504 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:09:51.551509 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:09:51.551792 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:09:51.551901 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 03:09:51.552818 (+ 917us) server_negotiation.cc:299] Negotiation successful
0812 03:09:51.552979 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":94,"threads_started":1}
WARNING: no policy specified for kudu/127.26.250.130@KRBTEST.COM; defaulting to no policy
I20260812 03:09:51.554451 7342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:51.535292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:35521 (local address 127.26.250.129:52467)
0812 03:09:51.535970 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0812 03:09:51.536031 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 03:09:51.537310 (+ 1279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:09:51.539462 (+ 2152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:09:51.539473 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:09:51.539941 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:09:51.540900 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:51.540927 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:51.542203 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:51.542210 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:51.543461 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:51.543472 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:51.544034 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:51.544039 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:51.544220 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:51.545061 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:09:51.545087 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:09:51.547365 (+ 2278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:09:51.550116 (+ 2751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:09:51.550125 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:09:51.550149 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:09:51.550683 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:09:51.551100 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:09:51.551116 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:09:51.551118 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:09:51.551354 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:09:51.551970 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:09:51.551978 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:09:51.552431 (+ 453us) client_negotiation.cc:769] Sending connection context
0812 03:09:51.552721 (+ 290us) client_negotiation.cc:240] Negotiation successful
0812 03:09:51.553202 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":167,"threads_started":1}
I20260812 03:09:51.556092 7340 heartbeater.cc:347] Connected to a master server at 127.26.250.190:35521
I20260812 03:09:51.556543 7340 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.130@KRBTEST.COM" created.
I20260812 03:09:51.557210 7340 heartbeater.cc:511] Master 127.26.250.190:35521 requested a full tablet report, sending...
I20260812 03:09:51.560576 7114 ts_manager.cc:194] Registered new tserver with Master: 1933f9c0ddd6409ea0855754142f3068 (127.26.250.129:34759)
I20260812 03:09:51.562529 7114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.129@KRBTEST.COM'} at 127.26.250.129:52467
Aug 12 03:09:51 dist-test-slave-zpfg krb5kdc[6598](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786504191, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
I20260812 03:09:51.579119 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.250.148:50291/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:0
--local_ip_for_outbound_sockets=127.26.250.130
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:35521
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:40681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:09:51.735697 7347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:09:51.736003 7347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:09:51.736048 7347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:09:51.740899 7347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:09:51.740981 7347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:09:51.741048 7347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:09:51.748333 7347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:40681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.26.250.148:50291/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:35521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:09:51.750093 7347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:09:51.751679 7347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:09:51.764057 7356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:09:51.764605 7354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:09:51.764088 7353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:09:51.765477 7347 server_base.cc:1034] running on GCE node
I20260812 03:09:51.766162 7347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:09:51.767081 7347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:09:51.770447 7347 hybrid_clock.cc:648] HybridClock initialized: now 1786504191770346 us; error 105 us; skew 500 ppm
Aug 12 03:09:51 dist-test-slave-zpfg krb5kdc[6598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504191, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:09:51.774019 7347 init.cc:377] Logged in from keytab as kudu/127.26.250.130@KRBTEST.COM (short username kudu)
I20260812 03:09:51.775245 7347 webserver.cc:460] Webserver started at http://127.26.250.130:45433/ using document root <none> and password file <none>
I20260812 03:09:51.775919 7347 fs_manager.cc:359] Metadata directory not provided
I20260812 03:09:51.776026 7347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:09:51.776335 7347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:09:51 dist-test-slave-zpfg krb5kdc[6598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504191, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for HTTP/127.26.250.148@KRBTEST.COM
I20260812 03:09:51.794765 7347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-1/data/instance:
uuid: "6aa80e2ed25f4e56adb82abbd704777e"
format_stamp: "Formatted at 2026-08-12 03:09:51 on dist-test-slave-zpfg"
server_key: "afc48ea153911134250d5b7307f4a758"
server_key_iv: "8a45b6655c87409d1bf90f25ca101cee"
server_key_version: "kuduclusterkey@0"
I20260812 03:09:51.796150 7347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance:
uuid: "6aa80e2ed25f4e56adb82abbd704777e"
format_stamp: "Formatted at 2026-08-12 03:09:51 on dist-test-slave-zpfg"
server_key: "afc48ea153911134250d5b7307f4a758"
server_key_iv: "8a45b6655c87409d1bf90f25ca101cee"
server_key_version: "kuduclusterkey@0"
I20260812 03:09:51.808475 7347 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.004s sys 0.000s
I20260812 03:09:51.841445 7363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:09:51.843566 7347 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.001s
I20260812 03:09:51.843758 7347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "6aa80e2ed25f4e56adb82abbd704777e"
format_stamp: "Formatted at 2026-08-12 03:09:51 on dist-test-slave-zpfg"
server_key: "afc48ea153911134250d5b7307f4a758"
server_key_iv: "8a45b6655c87409d1bf90f25ca101cee"
server_key_version: "kuduclusterkey@0"
I20260812 03:09:51.843957 7347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:09:51.867448 7347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:09:51.871672 7347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:09:51.871912 7347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:09:51.872818 7347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:09:51.874034 7347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:09:51.874109 7347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:09:51.874172 7347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:09:51.874240 7347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:09:51.887454 7347 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:42375
I20260812 03:09:51.888904 7347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:09:51.890130 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 7347
I20260812 03:09:51.890275 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance
I20260812 03:09:51.890681 27626 external_mini_cluster.cc:1428] Setting key 85eea48b79bb3b1e0f2771592dde8d72
I20260812 03:09:51.896261 7485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:42375 every 8 connection(s)
Aug 12 03:09:51 dist-test-slave-zpfg krb5kdc[6598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504191, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.250.131@KRBTEST.COM; defaulting to no policy
I20260812 03:09:51.909886 7343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:51.895360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:53521 (local address 127.26.250.190:35521)
0812 03:09:51.896709 (+ 1349us) server_negotiation.cc:206] Beginning negotiation
0812 03:09:51.896716 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:09:51.896736 (+ 20us) server_negotiation.cc:407] Connection header received
0812 03:09:51.896793 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:09:51.896798 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:09:51.896862 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:09:51.896964 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:09:51.898531 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:51.899496 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:51.901245 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:51.901560 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:51.901660 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:51.905121 (+ 3461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:09:51.905125 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:09:51.905127 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:09:51.905161 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:09:51.907194 (+ 2033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:09:51.907998 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:09:51.908002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:09:51.908004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:09:51.908102 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:09:51.908531 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:09:51.908534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:09:51.908538 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:09:51.908784 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:09:51.908895 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 03:09:51.909552 (+ 657us) server_negotiation.cc:299] Negotiation successful
0812 03:09:51.909706 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1221}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.131@KRBTEST.COM" created.
I20260812 03:09:51.910988 7489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:51.891945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:35521 (local address 127.26.250.130:53521)
0812 03:09:51.894804 (+ 2859us) negotiation.cc:107] Waiting for socket to connect
0812 03:09:51.894882 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0812 03:09:51.896184 (+ 1302us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:09:51.897219 (+ 1035us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:09:51.897231 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:09:51.897604 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:09:51.898318 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:51.898368 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:51.899676 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:51.899680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:51.901088 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:51.901101 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:51.901776 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:51.901781 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:51.902005 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:51.902790 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:09:51.902820 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:09:51.904877 (+ 2057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:09:51.907359 (+ 2482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:09:51.907367 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:09:51.907389 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:09:51.907850 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:09:51.908241 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:09:51.908243 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:09:51.908245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:09:51.908412 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:09:51.908943 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:09:51.908953 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:09:51.909269 (+ 316us) client_negotiation.cc:769] Sending connection context
0812 03:09:51.909456 (+ 187us) client_negotiation.cc:240] Negotiation successful
0812 03:09:51.909758 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2570,"thread_start_us":153,"threads_started":1}
I20260812 03:09:51.912443 7487 heartbeater.cc:347] Connected to a master server at 127.26.250.190:35521
I20260812 03:09:51.912734 7487 heartbeater.cc:464] Registering TS with master...
I20260812 03:09:51.913389 7487 heartbeater.cc:511] Master 127.26.250.190:35521 requested a full tablet report, sending...
I20260812 03:09:51.914770 7114 ts_manager.cc:194] Registered new tserver with Master: 6aa80e2ed25f4e56adb82abbd704777e (127.26.250.130:42375)
I20260812 03:09:51.915539 7114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.130@KRBTEST.COM'} at 127.26.250.130:53521
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
I20260812 03:09:51.946175 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.250.148:50291/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:0
--local_ip_for_outbound_sockets=127.26.250.131
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:35521
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:40681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:09:52 dist-test-slave-zpfg krb5kdc[6598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504191, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for HTTP/127.26.250.148@KRBTEST.COM
W20260812 03:09:52.127615 7492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:09:52.128153 7492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:09:52.128288 7492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:09:52.135111 7492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:09:52.135326 7492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:09:52.135484 7492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:09:52.144203 7492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:40681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.26.250.148:50291/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:35521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:09:52.145666 7492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:09:52.147109 7492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:09:52.158726 7498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:09:52.158739 7497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:09:52.159231 7500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:09:52.160473 7492 server_base.cc:1034] running on GCE node
I20260812 03:09:52.161095 7492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:09:52.162006 7492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:09:52.169458 7492 hybrid_clock.cc:648] HybridClock initialized: now 1786504192169357 us; error 114 us; skew 500 ppm
Aug 12 03:09:52 dist-test-slave-zpfg krb5kdc[6598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504192, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:09:52.173765 7492 init.cc:377] Logged in from keytab as kudu/127.26.250.131@KRBTEST.COM (short username kudu)
I20260812 03:09:52.175096 7492 webserver.cc:460] Webserver started at http://127.26.250.131:46125/ using document root <none> and password file <none>
I20260812 03:09:52.175880 7492 fs_manager.cc:359] Metadata directory not provided
I20260812 03:09:52.176004 7492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:09:52.176378 7492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:09:52 dist-test-slave-zpfg krb5kdc[6598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504192, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for HTTP/127.26.250.148@KRBTEST.COM
I20260812 03:09:52.199832 7492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-2/data/instance:
uuid: "bf89b6b6961e4d2e9afa3b8a3efc77a7"
format_stamp: "Formatted at 2026-08-12 03:09:52 on dist-test-slave-zpfg"
server_key: "d06713178c155ce477e740a87098d430"
server_key_iv: "043d15f0e2fca4f0f0da6d78ac13b779"
server_key_version: "kuduclusterkey@0"
I20260812 03:09:52.200709 7492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance:
uuid: "bf89b6b6961e4d2e9afa3b8a3efc77a7"
format_stamp: "Formatted at 2026-08-12 03:09:52 on dist-test-slave-zpfg"
server_key: "d06713178c155ce477e740a87098d430"
server_key_iv: "043d15f0e2fca4f0f0da6d78ac13b779"
server_key_version: "kuduclusterkey@0"
I20260812 03:09:52.207614 7492 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260812 03:09:52.236487 7508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:09:52.238575 7492 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260812 03:09:52.238739 7492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "bf89b6b6961e4d2e9afa3b8a3efc77a7"
format_stamp: "Formatted at 2026-08-12 03:09:52 on dist-test-slave-zpfg"
server_key: "d06713178c155ce477e740a87098d430"
server_key_iv: "043d15f0e2fca4f0f0da6d78ac13b779"
server_key_version: "kuduclusterkey@0"
I20260812 03:09:52.238910 7492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:09:52.273272 7492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:09:52.277755 7492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:09:52.277992 7492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:09:52.278823 7492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:09:52.280002 7492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:09:52.280079 7492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:09:52.280138 7492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:09:52.280224 7492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260812 03:09:52.295680 7492 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:37399
I20260812 03:09:52.297220 7492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:09:52.298445 7629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:37399 every 8 connection(s)
I20260812 03:09:52.302094 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 7492
I20260812 03:09:52.302248 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance
I20260812 03:09:52.302673 27626 external_mini_cluster.cc:1428] Setting key fa4d393da63f76ce5dcd6a825ab2fe1a
Aug 12 03:09:52 dist-test-slave-zpfg krb5kdc[6598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504192, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:09:52.316551 7343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:52.299843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:56151 (local address 127.26.250.190:35521)
0812 03:09:52.300016 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 03:09:52.300022 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:09:52.301319 (+ 1297us) server_negotiation.cc:407] Connection header received
0812 03:09:52.302748 (+ 1429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:09:52.302753 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:09:52.302817 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:09:52.302874 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:09:52.304764 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.305610 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.307520 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.307915 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.308056 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:52.311369 (+ 3313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:09:52.311374 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:09:52.311378 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:09:52.311414 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:09:52.313502 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:09:52.314301 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:09:52.314318 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:09:52.314321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:09:52.314483 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:09:52.315015 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:09:52.315019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:09:52.315021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:09:52.315354 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:09:52.315480 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 03:09:52.316219 (+ 739us) server_negotiation.cc:299] Negotiation successful
0812 03:09:52.316378 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 03:09:52.318655 7632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:52.300360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:35521 (local address 127.26.250.131:56151)
0812 03:09:52.301060 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0812 03:09:52.301131 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0812 03:09:52.302430 (+ 1299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:09:52.303412 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:09:52.303423 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:09:52.303823 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:09:52.304566 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:52.304586 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.305796 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.305799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:52.307360 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:52.307372 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.308116 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.308120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:52.308333 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:52.309118 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:09:52.309141 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:09:52.311103 (+ 1962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:09:52.313672 (+ 2569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:09:52.313679 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:09:52.313701 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:09:52.314133 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:09:52.314623 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:09:52.314627 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:09:52.314629 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:09:52.314867 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:09:52.315485 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:09:52.315493 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:09:52.315880 (+ 387us) client_negotiation.cc:769] Sending connection context
0812 03:09:52.317073 (+ 1193us) client_negotiation.cc:240] Negotiation successful
0812 03:09:52.317472 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":180,"threads_started":1}
I20260812 03:09:52.320392 7630 heartbeater.cc:347] Connected to a master server at 127.26.250.190:35521
I20260812 03:09:52.320685 7630 heartbeater.cc:464] Registering TS with master...
I20260812 03:09:52.321419 7630 heartbeater.cc:511] Master 127.26.250.190:35521 requested a full tablet report, sending...
I20260812 03:09:52.322813 7114 ts_manager.cc:194] Registered new tserver with Master: bf89b6b6961e4d2e9afa3b8a3efc77a7 (127.26.250.131:37399)
I20260812 03:09:52.323674 7114 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.131@KRBTEST.COM'} at 127.26.250.131:56151
I20260812 03:09:52.331960 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:09:52.345903 7343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:52.334808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47770 (local address 127.26.250.190:35521)
0812 03:09:52.334946 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 03:09:52.334951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:09:52.335210 (+ 259us) server_negotiation.cc:407] Connection header received
0812 03:09:52.335329 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:09:52.335333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:09:52.335388 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:09:52.335488 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:09:52.336456 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.337390 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.338927 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.339367 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.339460 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:52.341629 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:09:52.341634 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:09:52.341637 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:09:52.341670 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:09:52.343655 (+ 1985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:09:52.344264 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:09:52.344268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:09:52.344270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:09:52.344391 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:09:52.344749 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:09:52.344753 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:09:52.344755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:09:52.345006 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:09:52.345126 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 03:09:52.345574 (+ 448us) server_negotiation.cc:299] Negotiation successful
0812 03:09:52.345716 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 03:09:52.350600 7114 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47770:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:09:52.355039 7114 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:09:52.376441 7646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:52.366527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:37399 (local address 127.0.0.1:59108)
0812 03:09:52.368431 (+ 1904us) negotiation.cc:107] Waiting for socket to connect
0812 03:09:52.368490 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 03:09:52.368631 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:09:52.369941 (+ 1310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:09:52.369948 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:09:52.369970 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:09:52.370122 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:52.370128 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.371718 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.371722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:52.373081 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:52.373093 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.375830 (+ 2737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.375834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:52.375969 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:52.376020 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 03:09:52.376115 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 03:09:52.376276 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1715,"thread_start_us":107,"threads_started":1}
I20260812 03:09:52.377612 7643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:52.366704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59108 (local address 127.26.250.131:37399)
0812 03:09:52.369413 (+ 2709us) server_negotiation.cc:206] Beginning negotiation
0812 03:09:52.369424 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:09:52.369448 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:09:52.369524 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:09:52.369529 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:09:52.369764 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:09:52.370289 (+ 525us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:09:52.370457 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.371573 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.373500 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.374391 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.377135 (+ 2744us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:52.377304 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 03:09:52.377361 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 03:09:52.377468 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2597,"thread_start_us":132,"threads_started":1}
I20260812 03:09:52.380019 7644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:52.367919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:34759 (local address 127.0.0.1:55934)
0812 03:09:52.369096 (+ 1177us) negotiation.cc:107] Waiting for socket to connect
0812 03:09:52.369107 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:09:52.369165 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:09:52.372635 (+ 3470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:09:52.372639 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:09:52.372664 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:09:52.372781 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:52.372793 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.375611 (+ 2818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.375615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:52.376967 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:52.376980 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.378433 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.378437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:52.379672 (+ 1235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:52.379703 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 03:09:52.379769 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 03:09:52.379851 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1133,"thread_start_us":95,"threads_started":1}
I20260812 03:09:52.380829 7647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:52.368418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55934 (local address 127.26.250.129:34759)
0812 03:09:52.370554 (+ 2136us) server_negotiation.cc:206] Beginning negotiation
0812 03:09:52.370561 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:09:52.370587 (+ 26us) server_negotiation.cc:407] Connection header received
0812 03:09:52.370668 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:09:52.370674 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:09:52.372452 (+ 1778us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:09:52.372981 (+ 529us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:09:52.373148 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.375460 (+ 2312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.377183 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.378203 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.380281 (+ 2078us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:52.380477 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0812 03:09:52.380546 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 03:09:52.380663 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2016,"thread_start_us":87,"threads_started":1}
I20260812 03:09:52.382138 7642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:52.367748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:42375 (local address 127.0.0.1:45046)
0812 03:09:52.368876 (+ 1128us) negotiation.cc:107] Waiting for socket to connect
0812 03:09:52.368888 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 03:09:52.368957 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:09:52.373193 (+ 4236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:09:52.373196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:09:52.373216 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:09:52.373322 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:52.373326 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.377904 (+ 4578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.377907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:52.379073 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:52.379084 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.381743 (+ 2659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.381747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:52.381860 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:52.381885 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 03:09:52.381938 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 03:09:52.382009 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1068,"thread_start_us":135,"threads_started":1}
I20260812 03:09:52.383086 7645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:52.367917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45046 (local address 127.26.250.130:42375)
0812 03:09:52.370116 (+ 2199us) server_negotiation.cc:206] Beginning negotiation
0812 03:09:52.370136 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 03:09:52.370164 (+ 28us) server_negotiation.cc:407] Connection header received
0812 03:09:52.370253 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:09:52.370260 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:09:52.372057 (+ 1797us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:09:52.372244 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:09:52.374536 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.375653 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.379267 (+ 3614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.381580 (+ 2313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.382626 (+ 1046us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:52.382785 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 03:09:52.382839 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 03:09:52.382943 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2067,"thread_start_us":80,"threads_started":1}
I20260812 03:09:52.383975 7265 tablet_service.cc:1467] Processing CreateTablet for tablet a44195e786cf42ae946fe0f62615b460 (DEFAULT_TABLE table=test-table [id=6292a2dd8cab4564a69bb2636d42a6c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:09:52.383977 7553 tablet_service.cc:1467] Processing CreateTablet for tablet a44195e786cf42ae946fe0f62615b460 (DEFAULT_TABLE table=test-table [id=6292a2dd8cab4564a69bb2636d42a6c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:09:52.385393 7265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a44195e786cf42ae946fe0f62615b460. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:09:52.385988 7409 tablet_service.cc:1467] Processing CreateTablet for tablet a44195e786cf42ae946fe0f62615b460 (DEFAULT_TABLE table=test-table [id=6292a2dd8cab4564a69bb2636d42a6c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:09:52.387111 7409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a44195e786cf42ae946fe0f62615b460. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:09:52.387704 7553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a44195e786cf42ae946fe0f62615b460. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:09:52.396660 7648 tablet_bootstrap.cc:492] T a44195e786cf42ae946fe0f62615b460 P 6aa80e2ed25f4e56adb82abbd704777e: Bootstrap starting.
I20260812 03:09:52.398847 7650 tablet_bootstrap.cc:492] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7: Bootstrap starting.
I20260812 03:09:52.399278 7648 tablet_bootstrap.cc:654] T a44195e786cf42ae946fe0f62615b460 P 6aa80e2ed25f4e56adb82abbd704777e: Neither blocks nor log segments found. Creating new log.
I20260812 03:09:52.400791 7648 log.cc:824] T a44195e786cf42ae946fe0f62615b460 P 6aa80e2ed25f4e56adb82abbd704777e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:09:52.403322 7648 tablet_bootstrap.cc:492] T a44195e786cf42ae946fe0f62615b460 P 6aa80e2ed25f4e56adb82abbd704777e: No bootstrap required, opened a new log
I20260812 03:09:52.403558 7648 ts_tablet_manager.cc:1397] T a44195e786cf42ae946fe0f62615b460 P 6aa80e2ed25f4e56adb82abbd704777e: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260812 03:09:52.403712 7650 tablet_bootstrap.cc:654] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7: Neither blocks nor log segments found. Creating new log.
I20260812 03:09:52.404779 7650 log.cc:824] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7: Log is configured to *not* fsync() on all Append() calls
I20260812 03:09:52.405380 7649 tablet_bootstrap.cc:492] T a44195e786cf42ae946fe0f62615b460 P 1933f9c0ddd6409ea0855754142f3068: Bootstrap starting.
I20260812 03:09:52.407404 7650 tablet_bootstrap.cc:492] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7: No bootstrap required, opened a new log
I20260812 03:09:52.407641 7650 ts_tablet_manager.cc:1397] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 03:09:52.408211 7648 raft_consensus.cc:348] T a44195e786cf42ae946fe0f62615b460 P 6aa80e2ed25f4e56adb82abbd704777e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf89b6b6961e4d2e9afa3b8a3efc77a7" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 37399 } } peers { permanent_uuid: "6aa80e2ed25f4e56adb82abbd704777e" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 42375 } } peers { permanent_uuid: "1933f9c0ddd6409ea0855754142f3068" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34759 } }
I20260812 03:09:52.408521 7648 raft_consensus.cc:374] T a44195e786cf42ae946fe0f62615b460 P 6aa80e2ed25f4e56adb82abbd704777e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:09:52.408589 7648 raft_consensus.cc:729] T a44195e786cf42ae946fe0f62615b460 P 6aa80e2ed25f4e56adb82abbd704777e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6aa80e2ed25f4e56adb82abbd704777e, State: Initialized, Role: FOLLOWER
I20260812 03:09:52.409188 7648 consensus_queue.cc:260] T a44195e786cf42ae946fe0f62615b460 P 6aa80e2ed25f4e56adb82abbd704777e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf89b6b6961e4d2e9afa3b8a3efc77a7" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 37399 } } peers { permanent_uuid: "6aa80e2ed25f4e56adb82abbd704777e" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 42375 } } peers { permanent_uuid: "1933f9c0ddd6409ea0855754142f3068" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34759 } }
I20260812 03:09:52.410274 7648 ts_tablet_manager.cc:1428] T a44195e786cf42ae946fe0f62615b460 P 6aa80e2ed25f4e56adb82abbd704777e: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 03:09:52.410715 7487 heartbeater.cc:503] Master 127.26.250.190:35521 was elected leader, sending a full tablet report...
I20260812 03:09:52.411006 7649 tablet_bootstrap.cc:654] T a44195e786cf42ae946fe0f62615b460 P 1933f9c0ddd6409ea0855754142f3068: Neither blocks nor log segments found. Creating new log.
I20260812 03:09:52.411978 7649 log.cc:824] T a44195e786cf42ae946fe0f62615b460 P 1933f9c0ddd6409ea0855754142f3068: Log is configured to *not* fsync() on all Append() calls
I20260812 03:09:52.412325 7650 raft_consensus.cc:348] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf89b6b6961e4d2e9afa3b8a3efc77a7" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 37399 } } peers { permanent_uuid: "6aa80e2ed25f4e56adb82abbd704777e" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 42375 } } peers { permanent_uuid: "1933f9c0ddd6409ea0855754142f3068" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34759 } }
I20260812 03:09:52.412655 7650 raft_consensus.cc:374] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:09:52.412734 7650 raft_consensus.cc:729] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf89b6b6961e4d2e9afa3b8a3efc77a7, State: Initialized, Role: FOLLOWER
I20260812 03:09:52.414057 7650 consensus_queue.cc:260] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf89b6b6961e4d2e9afa3b8a3efc77a7" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 37399 } } peers { permanent_uuid: "6aa80e2ed25f4e56adb82abbd704777e" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 42375 } } peers { permanent_uuid: "1933f9c0ddd6409ea0855754142f3068" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34759 } }
I20260812 03:09:52.415802 7650 ts_tablet_manager.cc:1428] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260812 03:09:52.416253 7630 heartbeater.cc:503] Master 127.26.250.190:35521 was elected leader, sending a full tablet report...
I20260812 03:09:52.420389 7649 tablet_bootstrap.cc:492] T a44195e786cf42ae946fe0f62615b460 P 1933f9c0ddd6409ea0855754142f3068: No bootstrap required, opened a new log
I20260812 03:09:52.420665 7649 ts_tablet_manager.cc:1397] T a44195e786cf42ae946fe0f62615b460 P 1933f9c0ddd6409ea0855754142f3068: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.000s
I20260812 03:09:52.425805 7649 raft_consensus.cc:348] T a44195e786cf42ae946fe0f62615b460 P 1933f9c0ddd6409ea0855754142f3068 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf89b6b6961e4d2e9afa3b8a3efc77a7" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 37399 } } peers { permanent_uuid: "6aa80e2ed25f4e56adb82abbd704777e" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 42375 } } peers { permanent_uuid: "1933f9c0ddd6409ea0855754142f3068" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34759 } }
I20260812 03:09:52.426153 7649 raft_consensus.cc:374] T a44195e786cf42ae946fe0f62615b460 P 1933f9c0ddd6409ea0855754142f3068 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:09:52.426240 7649 raft_consensus.cc:729] T a44195e786cf42ae946fe0f62615b460 P 1933f9c0ddd6409ea0855754142f3068 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1933f9c0ddd6409ea0855754142f3068, State: Initialized, Role: FOLLOWER
I20260812 03:09:52.426851 7649 consensus_queue.cc:260] T a44195e786cf42ae946fe0f62615b460 P 1933f9c0ddd6409ea0855754142f3068 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf89b6b6961e4d2e9afa3b8a3efc77a7" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 37399 } } peers { permanent_uuid: "6aa80e2ed25f4e56adb82abbd704777e" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 42375 } } peers { permanent_uuid: "1933f9c0ddd6409ea0855754142f3068" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34759 } }
I20260812 03:09:52.428193 7649 ts_tablet_manager.cc:1428] T a44195e786cf42ae946fe0f62615b460 P 1933f9c0ddd6409ea0855754142f3068: Time spent starting tablet: real 0.007s user 0.009s sys 0.000s
I20260812 03:09:52.428604 7340 heartbeater.cc:503] Master 127.26.250.190:35521 was elected leader, sending a full tablet report...
Aug 12 03:09:52 dist-test-slave-zpfg krb5kdc[6598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504191, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for HTTP/127.26.250.148@KRBTEST.COM
I20260812 03:09:52.508814 7654 raft_consensus.cc:482] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:09:52.509057 7654 raft_consensus.cc:504] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf89b6b6961e4d2e9afa3b8a3efc77a7" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 37399 } } peers { permanent_uuid: "6aa80e2ed25f4e56adb82abbd704777e" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 42375 } } peers { permanent_uuid: "1933f9c0ddd6409ea0855754142f3068" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34759 } }
I20260812 03:09:52.510479 7654 leader_election.cc:288] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6aa80e2ed25f4e56adb82abbd704777e (127.26.250.130:42375), 1933f9c0ddd6409ea0855754142f3068 (127.26.250.129:34759)
I20260812 03:09:52.516312 7647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:52.511042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:40331 (local address 127.26.250.129:34759)
0812 03:09:52.511201 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 03:09:52.511206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:09:52.511991 (+ 785us) server_negotiation.cc:407] Connection header received
0812 03:09:52.512226 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:09:52.512229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:09:52.512277 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:09:52.512322 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:09:52.512691 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.513444 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.514958 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.515555 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.515650 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:52.515703 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 03:09:52.516061 (+ 358us) server_negotiation.cc:299] Negotiation successful
0812 03:09:52.516168 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 03:09:52.516485 7632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:52.510819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:42375 (local address 127.26.250.131:39031)
0812 03:09:52.511121 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0812 03:09:52.511135 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:09:52.511233 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:09:52.511980 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:09:52.511983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:09:52.512005 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:09:52.512140 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:52.512148 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.513345 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.513349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:52.514767 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:52.514779 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.515953 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.515957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:52.516070 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:52.516097 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 03:09:52.516215 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 03:09:52.516324 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":36}
I20260812 03:09:52.516752 7658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:52.510824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:34759 (local address 127.26.250.131:40331)
0812 03:09:52.511902 (+ 1078us) negotiation.cc:107] Waiting for socket to connect
0812 03:09:52.511915 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:09:52.512100 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:09:52.512455 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:09:52.512458 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:09:52.512473 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:09:52.512580 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:52.512585 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.513584 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.513587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:52.514811 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:09:52.514822 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.515814 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.515818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:09:52.515921 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:52.515942 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 03:09:52.516539 (+ 597us) client_negotiation.cc:240] Negotiation successful
0812 03:09:52.516613 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1018,"mutex_wait_us":658,"spinlock_wait_cycles":21248,"thread_start_us":140,"threads_started":1}
I20260812 03:09:52.517383 7276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a44195e786cf42ae946fe0f62615b460" candidate_uuid: "bf89b6b6961e4d2e9afa3b8a3efc77a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1933f9c0ddd6409ea0855754142f3068" is_pre_election: true
I20260812 03:09:52.517554 7645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:52.511398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:39031 (local address 127.26.250.130:42375)
0812 03:09:52.511585 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 03:09:52.511592 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:09:52.511614 (+ 22us) server_negotiation.cc:407] Connection header received
0812 03:09:52.511680 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:09:52.511684 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:09:52.511741 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:09:52.511825 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:09:52.512298 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.513172 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.514973 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.515675 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.515805 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:52.515874 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 03:09:52.517325 (+ 1451us) server_negotiation.cc:299] Negotiation successful
0812 03:09:52.517424 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 03:09:52.517727 7276 raft_consensus.cc:2454] T a44195e786cf42ae946fe0f62615b460 P 1933f9c0ddd6409ea0855754142f3068 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf89b6b6961e4d2e9afa3b8a3efc77a7 in term 0.
I20260812 03:09:52.518301 7512 leader_election.cc:302] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1933f9c0ddd6409ea0855754142f3068, bf89b6b6961e4d2e9afa3b8a3efc77a7; no voters:
I20260812 03:09:52.518445 7434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a44195e786cf42ae946fe0f62615b460" candidate_uuid: "bf89b6b6961e4d2e9afa3b8a3efc77a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6aa80e2ed25f4e56adb82abbd704777e" is_pre_election: true
I20260812 03:09:52.518728 7654 raft_consensus.cc:2781] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:09:52.518746 7434 raft_consensus.cc:2454] T a44195e786cf42ae946fe0f62615b460 P 6aa80e2ed25f4e56adb82abbd704777e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf89b6b6961e4d2e9afa3b8a3efc77a7 in term 0.
I20260812 03:09:52.518858 7654 raft_consensus.cc:482] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:09:52.518926 7654 raft_consensus.cc:3037] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:09:52.520515 7654 raft_consensus.cc:504] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf89b6b6961e4d2e9afa3b8a3efc77a7" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 37399 } } peers { permanent_uuid: "6aa80e2ed25f4e56adb82abbd704777e" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 42375 } } peers { permanent_uuid: "1933f9c0ddd6409ea0855754142f3068" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34759 } }
I20260812 03:09:52.520965 7654 leader_election.cc:288] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7 [CANDIDATE]: Term 1 election: Requested vote from peers 6aa80e2ed25f4e56adb82abbd704777e (127.26.250.130:42375), 1933f9c0ddd6409ea0855754142f3068 (127.26.250.129:34759)
I20260812 03:09:52.521394 7276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a44195e786cf42ae946fe0f62615b460" candidate_uuid: "bf89b6b6961e4d2e9afa3b8a3efc77a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1933f9c0ddd6409ea0855754142f3068"
I20260812 03:09:52.521427 7434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a44195e786cf42ae946fe0f62615b460" candidate_uuid: "bf89b6b6961e4d2e9afa3b8a3efc77a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6aa80e2ed25f4e56adb82abbd704777e"
I20260812 03:09:52.521530 7276 raft_consensus.cc:3037] T a44195e786cf42ae946fe0f62615b460 P 1933f9c0ddd6409ea0855754142f3068 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:09:52.521564 7434 raft_consensus.cc:3037] T a44195e786cf42ae946fe0f62615b460 P 6aa80e2ed25f4e56adb82abbd704777e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:09:52.523043 7276 raft_consensus.cc:2454] T a44195e786cf42ae946fe0f62615b460 P 1933f9c0ddd6409ea0855754142f3068 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf89b6b6961e4d2e9afa3b8a3efc77a7 in term 1.
I20260812 03:09:52.523097 7434 raft_consensus.cc:2454] T a44195e786cf42ae946fe0f62615b460 P 6aa80e2ed25f4e56adb82abbd704777e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf89b6b6961e4d2e9afa3b8a3efc77a7 in term 1.
I20260812 03:09:52.523542 7512 leader_election.cc:302] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1933f9c0ddd6409ea0855754142f3068, bf89b6b6961e4d2e9afa3b8a3efc77a7; no voters:
I20260812 03:09:52.523794 7654 raft_consensus.cc:2781] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:09:52.524032 7654 raft_consensus.cc:686] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7 [term 1 LEADER]: Becoming Leader. State: Replica: bf89b6b6961e4d2e9afa3b8a3efc77a7, State: Running, Role: LEADER
I20260812 03:09:52.524426 7654 consensus_queue.cc:237] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf89b6b6961e4d2e9afa3b8a3efc77a7" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 37399 } } peers { permanent_uuid: "6aa80e2ed25f4e56adb82abbd704777e" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 42375 } } peers { permanent_uuid: "1933f9c0ddd6409ea0855754142f3068" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34759 } }
W20260812 03:09:52.534891 7341 tablet.cc:2369] T a44195e786cf42ae946fe0f62615b460 P 1933f9c0ddd6409ea0855754142f3068: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:09:52.539110 7109 catalog_manager.cc:5495] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf89b6b6961e4d2e9afa3b8a3efc77a7 (127.26.250.131). New cstate: current_term: 1 leader_uuid: "bf89b6b6961e4d2e9afa3b8a3efc77a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf89b6b6961e4d2e9afa3b8a3efc77a7" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 37399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6aa80e2ed25f4e56adb82abbd704777e" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 42375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1933f9c0ddd6409ea0855754142f3068" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34759 } health_report { overall_health: UNKNOWN } } }
W20260812 03:09:52.549680 7631 tablet.cc:2369] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:09:52.585760 7643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:52.579387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59118 (local address 127.26.250.131:37399)
0812 03:09:52.579542 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 03:09:52.579547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:09:52.579659 (+ 112us) server_negotiation.cc:407] Connection header received
0812 03:09:52.579899 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:09:52.579902 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:09:52.579988 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:09:52.580121 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:09:52.580565 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.581411 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.583171 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:09:52.583547 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:09:52.583669 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:09:52.584214 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:09:52.584892 (+ 678us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:09:52.585466 (+ 574us) server_negotiation.cc:1035] Waiting for connection context
0812 03:09:52.585511 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 03:09:52.585590 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:09:52.593796 7434 raft_consensus.cc:1261] T a44195e786cf42ae946fe0f62615b460 P 6aa80e2ed25f4e56adb82abbd704777e [term 1 FOLLOWER]: Refusing update from remote peer bf89b6b6961e4d2e9afa3b8a3efc77a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:09:52.593873 7276 raft_consensus.cc:1261] T a44195e786cf42ae946fe0f62615b460 P 1933f9c0ddd6409ea0855754142f3068 [term 1 FOLLOWER]: Refusing update from remote peer bf89b6b6961e4d2e9afa3b8a3efc77a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:09:52.594659 7654 consensus_queue.cc:1035] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1933f9c0ddd6409ea0855754142f3068" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 34759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:09:52.595193 7654 consensus_queue.cc:1035] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6aa80e2ed25f4e56adb82abbd704777e" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 42375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:09:52.604403 7663 mvcc.cc:205] Tried to move back new op lower bound from 7317521172853243904 to 7317521172583284736. Current Snapshot: MvccSnapshot[applied={T|T < 7317521172853243904}]
I20260812 03:09:52.616724 7664 mvcc.cc:205] Tried to move back new op lower bound from 7317521172853243904 to 7317521172583284736. Current Snapshot: MvccSnapshot[applied={T|T < 7317521172853243904}]
I20260812 03:09:52.620322 7109 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:09:52.620551 7109 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:09:52.623420 7109 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0d6610f5a38442c99555510cba878ef0: Sending DeleteTablet for 3 replicas of tablet a44195e786cf42ae946fe0f62615b460
I20260812 03:09:52.623704 7665 mvcc.cc:205] Tried to move back new op lower bound from 7317521172853243904 to 7317521172583284736. Current Snapshot: MvccSnapshot[applied={T|T < 7317521172853243904}]
I20260812 03:09:52.624461 7409 tablet_service.cc:1514] Processing DeleteTablet for tablet a44195e786cf42ae946fe0f62615b460 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:09:52 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:45046
I20260812 03:09:52.624837 7553 tablet_service.cc:1514] Processing DeleteTablet for tablet a44195e786cf42ae946fe0f62615b460 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:09:52 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:59108
I20260812 03:09:52.625382 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 7202
I20260812 03:09:52.635048 7674 tablet_replica.cc:330] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7: stopping tablet replica
I20260812 03:09:52.635526 7674 raft_consensus.cc:2229] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:09:52.636010 7674 raft_consensus.cc:2258] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:09:52.640168 7674 ts_tablet_manager.cc:1905] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 03:09:52.643368 7100 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.250.129:34759 (error 108)
I20260812 03:09:52.643785 7673 tablet_replica.cc:330] T a44195e786cf42ae946fe0f62615b460 P 6aa80e2ed25f4e56adb82abbd704777e: stopping tablet replica
I20260812 03:09:52.644268 7673 raft_consensus.cc:2229] T a44195e786cf42ae946fe0f62615b460 P 6aa80e2ed25f4e56adb82abbd704777e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:09:52.644429 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 7347
I20260812 03:09:52.644775 7673 raft_consensus.cc:2258] T a44195e786cf42ae946fe0f62615b460 P 6aa80e2ed25f4e56adb82abbd704777e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:09:52.651607 7642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:09:52.651283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:34759 (local address 127.0.0.1:55946)
0812 03:09:52.651411 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 03:09:52.651500 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.129:34759: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 03:09:52.652220 7100 catalog_manager.cc:4584] TS 1933f9c0ddd6409ea0855754142f3068 (127.26.250.129:34759): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a44195e786cf42ae946fe0f62615b460: Network error: Client connection negotiation failed: client connection to 127.26.250.129:34759: connect: Connection refused (error 111)
I20260812 03:09:52.653393 7674 ts_tablet_manager.cc:1918] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:09:52.653568 7674 log.cc:1196] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/a44195e786cf42ae946fe0f62615b460
I20260812 03:09:52.654072 7674 ts_tablet_manager.cc:1939] T a44195e786cf42ae946fe0f62615b460 P bf89b6b6961e4d2e9afa3b8a3efc77a7: Deleting consensus metadata
I20260812 03:09:52.655740 7100 catalog_manager.cc:4841] TS bf89b6b6961e4d2e9afa3b8a3efc77a7 (127.26.250.131:37399): tablet a44195e786cf42ae946fe0f62615b460 (table test-table [id=6292a2dd8cab4564a69bb2636d42a6c5]) successfully deleted
W20260812 03:09:52.660377 7642 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:09:52.659851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:42375 (local address 127.0.0.1:45056)
0812 03:09:52.660000 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 03:09:52.660019 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:09:52.660133 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:09:52.660249 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.130:42375: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 03:09:52.660615 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 7492
I20260812 03:09:52.675575 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 7081
I20260812 03:09:52.686506 27626 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 03:09:53.032471 27626 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 03:09:53.032615 27626 mini_ranger.cc:66] Stopping Ranger...
I20260812 03:09:53.389051 27626 mini_ranger.cc:68] Stopped Ranger
2026-08-12 03:09:53.389 UTC [6630] LOG: received smart shutdown request
2026-08-12 03:09:53.394 UTC [6630] LOG: background worker "logical replication launcher" (PID 6640) exited with exit code 1
2026-08-12 03:09:53.394 UTC [6635] LOG: shutting down
2026-08-12 03:09:53.644 UTC [6630] LOG: database system is shut down
2026-08-12T03:09:53Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (53451 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:09:53 dist-test-slave-zpfg krb5kdc[7684](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:09:53 dist-test-slave-zpfg krb5kdc[7684](info): set up 2 sockets
Aug 12 03:09:53 dist-test-slave-zpfg krb5kdc[7684](info): commencing operation
krb5kdc: starting...
W20260812 03:09:54.885061 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:09:54 dist-test-slave-zpfg krb5kdc[7684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:09:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:09:54Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.26.250.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.26.250.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.26.250.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/rangeradmin_127.26.250.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.26.250.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/rangeradmin_127.26.250.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.26.250.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.26.250.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.26.250.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/rangerlookup_127.26.250.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.26.250.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/rangerlookup_127.26.250.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.250.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.26.250.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.250.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/HTTP_127.26.250.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.250.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/HTTP_127.26.250.148@KRBTEST.COM.keytab.
I20260812 03:09:55.082758 27626 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskh6ikwC/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/postgres -l logfile start
2026-08-12 03:09:58.175 UTC [7716] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 03:09:58.175 UTC [7716] LOG: listening on IPv4 address "127.26.250.148", port 43699
2026-08-12 03:09:58.180 UTC [7716] LOG: listening on Unix socket "/tmp/.s.PGSQL.43699"
2026-08-12 03:09:58.203 UTC [7719] LOG: database system was shut down at 2026-08-12 03:09:56 UTC
2026-08-12 03:09:58.208 UTC [7716] LOG: database system is ready to accept connections
I20260812 03:09:59.224895 27626 mini_postgres.cc:96] Postgres bound to 43699
2026-08-12 03:09:59.233 UTC [7727] FATAL: database "slave" does not exist
127.26.250.148:43699 - accepting connections
I20260812 03:09:59.233634 27626 mini_ranger.cc:161] Starting Ranger...
I20260812 03:09:59.251147 27626 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 03:09:59.817333 27626 mini_ranger.cc:87] Created ranger Postgres database
I20260812 03:09:59.817459 27626 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ranger-admin
2026-08-12 03:10:00,112 [I] DB FLAVOR :POSTGRES
2026-08-12 03:10:00,112 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 03:10:00,112 [I] Checking connection..
2026-08-12 03:10:00,113 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:43699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 03:10:00,377 [I] Checking connection passed.
2026-08-12 03:10:00,378 [I] --------- Verifying version history table ---------
2026-08-12 03:10:00,378 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:43699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:10:00,629 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 03:10:00,629 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:43699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:10:00,880 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 03:10:00,880 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 03:10:00,880 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:43699/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 03:10:01.160 UTC [7794] WARNING: there is no transaction in progress
2026-08-12 03:10:01,169 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 03:10:01,169 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:43699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 03:10:01,443 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 03:10:01,443 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 03:10:01,444 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:43699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 03:10:01,715 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:43699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 03:10:01,990 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:43699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zpfg.c.gcp-upstream.internal','N') ;"
2026-08-12 03:10:02,276 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:43699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 03:10:02,559 [I] Table x_portal_user does not exist in database ranger
2026-08-12 03:10:02,559 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:43699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 03:10:02,819 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 03:10:02,819 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:43699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 03:10:03,097 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:43699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 03:10:03,355 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 03:10:03,355 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:43699/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 03:10:04.511 UTC [7904] WARNING: there is no transaction in progress
2026-08-12 03:10:04.530 UTC [7904] WARNING: there is no transaction in progress
2026-08-12 03:10:04.550 UTC [7904] WARNING: there is no transaction in progress
2026-08-12 03:10:04.568 UTC [7904] WARNING: there is no transaction in progress
2026-08-12 03:10:04.588 UTC [7904] WARNING: there is no transaction in progress
2026-08-12 03:10:04.609 UTC [7904] WARNING: there is no transaction in progress
2026-08-12 03:10:04.675 UTC [7904] WARNING: there is no transaction in progress
2026-08-12 03:10:04.683 UTC [7904] WARNING: there is no transaction in progress
2026-08-12 03:10:04.699 UTC [7904] WARNING: there is no transaction in progress
2026-08-12 03:10:04.711 UTC [7904] WARNING: there is no transaction in progress
2026-08-12 03:10:04.724 UTC [7904] WARNING: there is no transaction in progress
2026-08-12 03:10:04.739 UTC [7904] WARNING: there is no transaction in progress
2026-08-12 03:10:04.751 UTC [7904] WARNING: there is no transaction in progress
2026-08-12 03:10:04.759 UTC [7904] WARNING: there is no transaction in progress
2026-08-12 03:10:04.767 UTC [7904] WARNING: there is no transaction in progress
2026-08-12 03:10:04.775 UTC [7904] WARNING: there is no transaction in progress
2026-08-12 03:10:05,575 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 03:10:05,576 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:43699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 03:10:05,848 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 03:10:05,849 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:43699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 03:10:06,107 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 03:10:06,107 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:43699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 03:10:06,378 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 03:10:06,379 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:43699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 03:10:06,894 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:43699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 03:10:07,419 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:43699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zpfg.c.gcp-upstream.internal';"
2026-08-12 03:10:07,683 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 03:10:07,683 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:43699/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 03:10:07,953 [I] DB_PATCHES have already been applied
I20260812 03:10:07.960532 27626 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ranger-admin:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskh6ikwC/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 03:10:07.960609 27626 mini_ranger.cc:193] Using host: 127.26.250.148
I20260812 03:10:07.965385 27626 mini_ranger.cc:227] Ranger admin URL: http://127.26.250.148:42855
Aug 12, 2026 3:10:08 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 3:10:08 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 3:10:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 3:10:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 3:10:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 3:10:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 3:10:08 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 03:10:09.156184 27626 mini_ranger.cc:160] Time spent starting Ranger: real 9.923s user 0.001s sys 0.013s
Aug 12, 2026 3:10:30 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 3:10:30 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Aug 12, 2026 3:10:30 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 3:10:30 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 3:10:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 3:10:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 3:10:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 3:10:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 3:10:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 3:10:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 3:10:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 3:10:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 3:10:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 3:10:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 3:10:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 3:10:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 3:10:30 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 3:10:31 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 03:10:32.222154 27626 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.26.250.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.26.250.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.26.250.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/rangerkms_127.26.250.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.26.250.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/rangerkms_127.26.250.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.26.250.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.26.250.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.26.250.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/HTTP_127.26.250.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.26.250.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/HTTP_127.26.250.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 03:10:32 dist-test-slave-zpfg krb5kdc[7684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504232, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 03:10:32.324684 27626 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 03:10:32.337297 27626 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 03:10:33.408732 27626 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 03:10:33.408841 27626 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ranger-kms
I20260812 03:10:33.408867 27626 mini_ranger_kms.cc:227] Using postgres at 127.26.250.148:43699
2026-08-12 03:10:33,620 [I] DB FLAVOR :POSTGRES
2026-08-12 03:10:33,621 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 03:10:33,621 [I] Checking connection
2026-08-12 03:10:33,621 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:43699/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 03:10:33,895 [I] connection success
2026-08-12 03:10:33,896 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 03:10:33,896 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 03:10:33,896 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:43699/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 03:10:34,206 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 03:10:34,206 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 03:10:34,207 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 03:10:34,207 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.26.250.148:43699/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 03:10:34,508 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 03:10:35.031803 27626 mini_ranger_kms.cc:310] Created kms service
I20260812 03:10:35.262925 27626 mini_ranger_kms.cc:326] Created kudu user
I20260812 03:10:35.315383 27626 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 03:10:35 dist-test-slave-zpfg krb5kdc[7684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504232, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.26.250.148@KRBTEST.COM
I20260812 03:10:35.743152 27626 mini_ranger_kms.cc:384] Added ranger policy
I20260812 03:10:35.743311 27626 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ranger-kms:/tmp/dist-test-taskh6ikwC/build/debug/bin/postgresql.jar:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskh6ikwC/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 03:10:35.743333 27626 mini_ranger_kms.cc:240] Using host: 127.26.250.148
I20260812 03:10:35.748085 27626 mini_ranger_kms.cc:276] Ranger KMS PID: 8100
I20260812 03:10:35.748188 27626 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.26.250.148:43877
Aug 12, 2026 3:10:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 3:10:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 3:10:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 3:10:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 3:10:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 3:10:37 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 03:10:38 dist-test-slave-zpfg krb5kdc[7684](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786504238, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.250.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:10:38.144501 27626 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.820s user 0.011s sys 0.177s
I20260812 03:10:38.144651 27626 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 03:10:38.144750 27626 mini_ranger_kms.cc:401] 127.26.250.148:43877/kms/v1/keys
Aug 12 03:10:38 dist-test-slave-zpfg krb5kdc[7684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504238, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.250.148@KRBTEST.COM for HTTP/127.26.250.148@KRBTEST.COM
Aug 12 03:10:38 dist-test-slave-zpfg krb5kdc[7684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504238, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.26.250.148@KRBTEST.COM for HTTP/127.26.250.148@KRBTEST.COM
Aug 12 03:10:41 dist-test-slave-zpfg krb5kdc[7684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:10:41.485963 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:39327
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:39327
--ranger_config_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.250.148:43877/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:10:41.622186 8150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:10:41.622601 8150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:10:41.622684 8150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:10:41.627215 8150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:10:41.627290 8150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:10:41.627310 8150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:10:41.627327 8150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:10:41.627343 8150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:10:41.633234 8150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/wal
--ranger_kms_url=127.26.250.148:43877/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:39327
--ranger_config_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:39327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:10:41.634554 8150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:10:41.635654 8150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:10:41.648129 8156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:41.648305 8150 server_base.cc:1034] running on GCE node
W20260812 03:10:41.648329 8155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:41.648545 8158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:41.649524 8150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:10:41.651180 8150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:10:41.652407 8150 hybrid_clock.cc:648] HybridClock initialized: now 1786504241652361 us; error 59 us; skew 500 ppm
Aug 12 03:10:41 dist-test-slave-zpfg krb5kdc[7684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504241, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:10:41.657346 8150 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:10:41.658592 8150 webserver.cc:460] Webserver started at http://127.26.250.190:43541/ using document root <none> and password file <none>
I20260812 03:10:41.659276 8150 fs_manager.cc:359] Metadata directory not provided
I20260812 03:10:41.659344 8150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:10:41.659601 8150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:10:41 dist-test-slave-zpfg krb5kdc[7684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504241, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for HTTP/127.26.250.148@KRBTEST.COM
I20260812 03:10:41.716984 8150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "dc9655f1863e4ed79f613d57291d889e"
format_stamp: "Formatted at 2026-08-12 03:10:41 on dist-test-slave-zpfg"
server_key: "b228b9b9540b2e602fcd1ae96b00e7b5"
server_key_iv: "37f3af53ac735090abdf7345f3a8b3ed"
server_key_version: "kuduclusterkey@0"
I20260812 03:10:41.717731 8150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "dc9655f1863e4ed79f613d57291d889e"
format_stamp: "Formatted at 2026-08-12 03:10:41 on dist-test-slave-zpfg"
server_key: "b228b9b9540b2e602fcd1ae96b00e7b5"
server_key_iv: "37f3af53ac735090abdf7345f3a8b3ed"
server_key_version: "kuduclusterkey@0"
I20260812 03:10:41.722298 8150 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 03:10:41.739851 8165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:41.741426 8150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:10:41.741555 8150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "dc9655f1863e4ed79f613d57291d889e"
format_stamp: "Formatted at 2026-08-12 03:10:41 on dist-test-slave-zpfg"
server_key: "b228b9b9540b2e602fcd1ae96b00e7b5"
server_key_iv: "37f3af53ac735090abdf7345f3a8b3ed"
server_key_version: "kuduclusterkey@0"
I20260812 03:10:41.741683 8150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:10:41.764811 8150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:10:41.768291 8150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:10:41.768479 8150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:10:41.776538 8150 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:39327
I20260812 03:10:41.776674 8221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:39327 every 8 connection(s)
I20260812 03:10:41.777676 8150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:10:41.780880 8222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:10:41.786803 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 8150
I20260812 03:10:41.786998 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:10:41.787222 8222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e: Bootstrap starting.
I20260812 03:10:41.787324 27626 external_mini_cluster.cc:1428] Setting key 980293937e21044a05e730c3412acd9f
I20260812 03:10:41.790041 8222 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e: Neither blocks nor log segments found. Creating new log.
I20260812 03:10:41.791014 8222 log.cc:824] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:10:41.793481 8222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e: No bootstrap required, opened a new log
Aug 12 03:10:41 dist-test-slave-zpfg krb5kdc[7684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:10:41.798161 8222 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9655f1863e4ed79f613d57291d889e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39327 } }
I20260812 03:10:41.798501 8222 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:10:41.798586 8222 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9655f1863e4ed79f613d57291d889e, State: Initialized, Role: FOLLOWER
I20260812 03:10:41.799201 8222 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9655f1863e4ed79f613d57291d889e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39327 } }
I20260812 03:10:41.799399 8222 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:10:41.799500 8222 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:10:41.799613 8222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:10:41.800629 8222 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9655f1863e4ed79f613d57291d889e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39327 } }
I20260812 03:10:41.801008 8222 leader_election.cc:302] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc9655f1863e4ed79f613d57291d889e; no voters:
I20260812 03:10:41.801429 8222 leader_election.cc:288] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:10:41.801556 8228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:10:41.801810 8228 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [term 1 LEADER]: Becoming Leader. State: Replica: dc9655f1863e4ed79f613d57291d889e, State: Running, Role: LEADER
I20260812 03:10:41.802172 8228 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9655f1863e4ed79f613d57291d889e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39327 } }
I20260812 03:10:41.802892 8222 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:10:41.804293 8226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:41.788928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42546 (local address 127.26.250.190:39327)
0812 03:10:41.789464 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:41.789482 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:41.789520 (+ 38us) server_negotiation.cc:407] Connection header received
0812 03:10:41.790233 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:41.790266 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:41.790701 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:41.791005 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:41.792064 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:41.793087 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:41.794196 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:41.794773 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:41.794985 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:41.797998 (+ 3013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:41.798005 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:41.798021 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:41.798053 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:41.800250 (+ 2197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:41.800755 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:41.800773 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:41.800779 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:41.800921 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:41.801278 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:41.801285 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:41.801288 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:41.801870 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:41.802714 (+ 844us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:41.802882 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 03:10:41.803176 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":161,"threads_started":1}
I20260812 03:10:41.804242 8230 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc9655f1863e4ed79f613d57291d889e. Latest consensus state: current_term: 1 leader_uuid: "dc9655f1863e4ed79f613d57291d889e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9655f1863e4ed79f613d57291d889e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39327 } } }
I20260812 03:10:41.804697 8230 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [sys.catalog]: This master's current role is: LEADER
I20260812 03:10:41.804520 8229 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc9655f1863e4ed79f613d57291d889e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9655f1863e4ed79f613d57291d889e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39327 } } }
I20260812 03:10:41.804991 8229 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [sys.catalog]: This master's current role is: LEADER
I20260812 03:10:41.805287 8222 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 03:10:41.809334 8242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:10:41.814566 8242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:10:41.822770 8242 catalog_manager.cc:1347] Generated new cluster ID: bbaee1e38b29445e91fa90eb40d96e4b
I20260812 03:10:41.822927 8242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:10:41.851400 8242 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:10:41.852744 8242 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:10:41.861272 8242 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e: Generated new TSK 0
I20260812 03:10:41.862160 8242 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.250.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
I20260812 03:10:42.590355 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.250.148:43877/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:0
--local_ip_for_outbound_sockets=127.26.250.129
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39327
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:10:42.722293 8269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:10:42.722672 8269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:10:42.722797 8269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:10:42.727334 8269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:10:42.727430 8269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:10:42.727537 8269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:10:42.733549 8269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.26.250.148:43877/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:10:42.735061 8269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:10:42.736276 8269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:10:42.747015 8275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:42.747008 8277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:42.747031 8274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:42.747354 8269 server_base.cc:1034] running on GCE node
I20260812 03:10:42.747859 8269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:10:42.748785 8269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:10:42.749995 8269 hybrid_clock.cc:648] HybridClock initialized: now 1786504242749900 us; error 125 us; skew 500 ppm
Aug 12 03:10:42 dist-test-slave-zpfg krb5kdc[7684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504242, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:10:42.753795 8269 init.cc:377] Logged in from keytab as kudu/127.26.250.129@KRBTEST.COM (short username kudu)
I20260812 03:10:42.754983 8269 webserver.cc:460] Webserver started at http://127.26.250.129:35433/ using document root <none> and password file <none>
I20260812 03:10:42.755597 8269 fs_manager.cc:359] Metadata directory not provided
I20260812 03:10:42.755664 8269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:10:42.755898 8269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:10:42 dist-test-slave-zpfg krb5kdc[7684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504242, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for HTTP/127.26.250.148@KRBTEST.COM
I20260812 03:10:42.775414 8269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/data/instance:
uuid: "52363daed5bc43d0b15c2286fe6f1546"
format_stamp: "Formatted at 2026-08-12 03:10:42 on dist-test-slave-zpfg"
server_key: "8c8d5fd1481741cf80ef6303f9e148bb"
server_key_iv: "25e34ac47a2a62c282d3513d26dbfa79"
server_key_version: "kuduclusterkey@0"
I20260812 03:10:42.776284 8269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance:
uuid: "52363daed5bc43d0b15c2286fe6f1546"
format_stamp: "Formatted at 2026-08-12 03:10:42 on dist-test-slave-zpfg"
server_key: "8c8d5fd1481741cf80ef6303f9e148bb"
server_key_iv: "25e34ac47a2a62c282d3513d26dbfa79"
server_key_version: "kuduclusterkey@0"
I20260812 03:10:42.798310 8269 fs_manager.cc:691] Time spent creating directory manager: real 0.022s user 0.006s sys 0.000s
I20260812 03:10:42.815618 8286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:42.817507 8269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260812 03:10:42.817655 8269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "52363daed5bc43d0b15c2286fe6f1546"
format_stamp: "Formatted at 2026-08-12 03:10:42 on dist-test-slave-zpfg"
server_key: "8c8d5fd1481741cf80ef6303f9e148bb"
server_key_iv: "25e34ac47a2a62c282d3513d26dbfa79"
server_key_version: "kuduclusterkey@0"
I20260812 03:10:42.817796 8269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:10:42.836695 8269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:10:42.840655 8269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:10:42.840938 8269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:10:42.841717 8269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:10:42.842967 8269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:10:42.843091 8269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:42.843237 8269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:10:42.843351 8269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:42.855115 8269 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:33039
I20260812 03:10:42.855201 8404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:33039 every 8 connection(s)
I20260812 03:10:42.856148 8269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:10:42.856671 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 8269
I20260812 03:10:42.856812 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance
I20260812 03:10:42.857160 27626 external_mini_cluster.cc:1428] Setting key a6a775fb623d6be5aac54929d3cb6291
Aug 12 03:10:42 dist-test-slave-zpfg krb5kdc[7684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504242, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.250.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.130@KRBTEST.COM" created.
I20260812 03:10:42.875229 8407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:42.858088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:47721 (local address 127.26.250.190:39327)
0812 03:10:42.858682 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:42.858691 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:42.859422 (+ 731us) server_negotiation.cc:407] Connection header received
0812 03:10:42.860795 (+ 1373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:42.860798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:42.860858 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:42.860958 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:42.862707 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:42.863597 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:42.865606 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:42.865991 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:42.866124 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:42.869757 (+ 3633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:42.869762 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:42.869765 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:42.869802 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:42.871699 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:42.872393 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:42.872397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:42.872399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:42.872499 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:42.872938 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:42.872942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:42.872943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:42.873197 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:42.873307 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:42.874750 (+ 1443us) server_negotiation.cc:299] Negotiation successful
0812 03:10:42.874926 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":117,"threads_started":1}
I20260812 03:10:42.877502 8408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:42.858726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39327 (local address 127.26.250.129:47721)
0812 03:10:42.859269 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:42.859327 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:42.860339 (+ 1012us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:42.861176 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:42.861188 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:42.861622 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:10:42.862483 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:42.862512 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:42.863790 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:42.863796 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:42.865438 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:42.865454 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:42.866203 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:42.866211 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:42.866561 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:42.867385 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:10:42.867416 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:10:42.869492 (+ 2076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:10:42.871842 (+ 2350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:42.871850 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:42.871870 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:42.872269 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:42.872639 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:42.872653 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:42.872655 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:42.872819 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:42.873901 (+ 1082us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:10:42.873908 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:10:42.874409 (+ 501us) client_negotiation.cc:769] Sending connection context
0812 03:10:42.875824 (+ 1415us) client_negotiation.cc:240] Negotiation successful
0812 03:10:42.876285 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":149,"threads_started":1}
I20260812 03:10:42.879489 8405 heartbeater.cc:347] Connected to a master server at 127.26.250.190:39327
I20260812 03:10:42.879763 8405 heartbeater.cc:464] Registering TS with master...
I20260812 03:10:42.880422 8405 heartbeater.cc:511] Master 127.26.250.190:39327 requested a full tablet report, sending...
I20260812 03:10:42.882550 8182 ts_manager.cc:194] Registered new tserver with Master: 52363daed5bc43d0b15c2286fe6f1546 (127.26.250.129:33039)
I20260812 03:10:42.884240 8182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.129@KRBTEST.COM'} at 127.26.250.129:47721
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
I20260812 03:10:42.911012 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.250.148:43877/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:0
--local_ip_for_outbound_sockets=127.26.250.130
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39327
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:10:42 dist-test-slave-zpfg krb5kdc[7684](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786504242, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 03:10:43.077682 8413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:10:43.077997 8413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:10:43.078094 8413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:10:43.082556 8413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:10:43.082645 8413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:10:43.082757 8413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:10:43.089013 8413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.26.250.148:43877/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:10:43.090292 8413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:10:43.091499 8413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:10:43.102975 8420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:43.103078 8423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:43.103371 8419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:43.104776 8413 server_base.cc:1034] running on GCE node
I20260812 03:10:43.105366 8413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:10:43.106251 8413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:10:43.107455 8413 hybrid_clock.cc:648] HybridClock initialized: now 1786504243107420 us; error 54 us; skew 500 ppm
Aug 12 03:10:43 dist-test-slave-zpfg krb5kdc[7684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504243, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:10:43.111044 8413 init.cc:377] Logged in from keytab as kudu/127.26.250.130@KRBTEST.COM (short username kudu)
I20260812 03:10:43.112362 8413 webserver.cc:460] Webserver started at http://127.26.250.130:36671/ using document root <none> and password file <none>
I20260812 03:10:43.113127 8413 fs_manager.cc:359] Metadata directory not provided
I20260812 03:10:43.113240 8413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:10:43.113548 8413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:10:43 dist-test-slave-zpfg krb5kdc[7684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504243, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for HTTP/127.26.250.148@KRBTEST.COM
I20260812 03:10:43.131036 8413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/data/instance:
uuid: "e3fd0a8e70404793955c3b6f63f05d19"
format_stamp: "Formatted at 2026-08-12 03:10:43 on dist-test-slave-zpfg"
server_key: "677be9fbf72b8d7beba2d13e04419b7e"
server_key_iv: "30f58169b8fc94c366443618a8a9c7f6"
server_key_version: "kuduclusterkey@0"
I20260812 03:10:43.131807 8413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance:
uuid: "e3fd0a8e70404793955c3b6f63f05d19"
format_stamp: "Formatted at 2026-08-12 03:10:43 on dist-test-slave-zpfg"
server_key: "677be9fbf72b8d7beba2d13e04419b7e"
server_key_iv: "30f58169b8fc94c366443618a8a9c7f6"
server_key_version: "kuduclusterkey@0"
I20260812 03:10:43.140053 8413 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.000s
I20260812 03:10:43.163522 8430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:43.165232 8413 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:10:43.165410 8413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "e3fd0a8e70404793955c3b6f63f05d19"
format_stamp: "Formatted at 2026-08-12 03:10:43 on dist-test-slave-zpfg"
server_key: "677be9fbf72b8d7beba2d13e04419b7e"
server_key_iv: "30f58169b8fc94c366443618a8a9c7f6"
server_key_version: "kuduclusterkey@0"
I20260812 03:10:43.165575 8413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:10:43.209559 8413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:10:43.213721 8413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:10:43.213976 8413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:10:43.214849 8413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:10:43.216109 8413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:10:43.216212 8413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:43.216284 8413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:10:43.216317 8413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:43.227984 8413 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:46243
I20260812 03:10:43.228065 8547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:46243 every 8 connection(s)
I20260812 03:10:43.229269 8413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:10:43.237756 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 8413
I20260812 03:10:43.237895 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance
I20260812 03:10:43.238270 27626 external_mini_cluster.cc:1428] Setting key 4d51c3d1dd01a751c188fb142e6bb154
Aug 12 03:10:43 dist-test-slave-zpfg krb5kdc[7684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504243, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:10:43.247875 8407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:43.231800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:53461 (local address 127.26.250.190:39327)
0812 03:10:43.231980 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:43.231984 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:43.233047 (+ 1063us) server_negotiation.cc:407] Connection header received
0812 03:10:43.234313 (+ 1266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:43.234317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:43.234421 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:43.234519 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:43.235975 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:43.236621 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:43.238268 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:43.238659 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:43.238752 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:43.242464 (+ 3712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:43.242471 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:43.242477 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:43.242517 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:43.244612 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:43.245467 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:43.245474 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:43.245479 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:43.245582 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:43.246142 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:43.246148 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:43.246153 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:43.246475 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:43.246559 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:43.247452 (+ 893us) server_negotiation.cc:299] Negotiation successful
0812 03:10:43.247627 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 03:10:43.250252 8550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:43.232239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39327 (local address 127.26.250.130:53461)
0812 03:10:43.232844 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:43.232906 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:43.234003 (+ 1097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:43.234755 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:43.234764 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:43.235150 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:10:43.235796 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:43.235813 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:43.236779 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:43.236783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:43.238020 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:43.238036 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:43.239001 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:43.239005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:43.239268 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:43.240082 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:10:43.240106 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:10:43.242151 (+ 2045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:10:43.244815 (+ 2664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:43.244825 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:43.244848 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:43.245334 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:43.245717 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:43.245723 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:43.245725 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:43.246022 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:43.246648 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:10:43.246657 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:10:43.247102 (+ 445us) client_negotiation.cc:769] Sending connection context
0812 03:10:43.248391 (+ 1289us) client_negotiation.cc:240] Negotiation successful
0812 03:10:43.248923 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":178,"threads_started":1}
I20260812 03:10:43.252228 8548 heartbeater.cc:347] Connected to a master server at 127.26.250.190:39327
I20260812 03:10:43.252514 8548 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.26.250.131@KRBTEST.COM; defaulting to no policy
I20260812 03:10:43.253206 8548 heartbeater.cc:511] Master 127.26.250.190:39327 requested a full tablet report, sending...
I20260812 03:10:43.254747 8182 ts_manager.cc:194] Registered new tserver with Master: e3fd0a8e70404793955c3b6f63f05d19 (127.26.250.130:46243)
I20260812 03:10:43.255820 8182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.130@KRBTEST.COM'} at 127.26.250.130:53461
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
I20260812 03:10:43.282091 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.250.148:43877/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:0
--local_ip_for_outbound_sockets=127.26.250.131
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39327
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:10:43.451519 8553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:10:43.451922 8553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:10:43.451996 8553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:10:43.458858 8553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:10:43.458966 8553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:10:43.459056 8553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
Aug 12 03:10:43 dist-test-slave-zpfg krb5kdc[7684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504242, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for HTTP/127.26.250.148@KRBTEST.COM
I20260812 03:10:43.467700 8553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.26.250.148:43877/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:10:43.469182 8553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:10:43.470824 8553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:10:43.485495 8561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:43.485886 8559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:43.487574 8558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:43.487850 8553 server_base.cc:1034] running on GCE node
I20260812 03:10:43.489439 8553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:10:43.490406 8553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:10:43.498325 8553 hybrid_clock.cc:648] HybridClock initialized: now 1786504243497915 us; error 444 us; skew 500 ppm
Aug 12 03:10:43 dist-test-slave-zpfg krb5kdc[7684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504243, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:10:43.502733 8553 init.cc:377] Logged in from keytab as kudu/127.26.250.131@KRBTEST.COM (short username kudu)
I20260812 03:10:43.504391 8553 webserver.cc:460] Webserver started at http://127.26.250.131:39523/ using document root <none> and password file <none>
I20260812 03:10:43.505256 8553 fs_manager.cc:359] Metadata directory not provided
I20260812 03:10:43.505434 8553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:10:43.505772 8553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 03:10:43 dist-test-slave-zpfg krb5kdc[7684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504243, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for HTTP/127.26.250.148@KRBTEST.COM
I20260812 03:10:43.530959 8553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/data/instance:
uuid: "1a61de979223434992ead55ddce3c027"
format_stamp: "Formatted at 2026-08-12 03:10:43 on dist-test-slave-zpfg"
server_key: "6e8959c1e58e3de30ed699b57f214b67"
server_key_iv: "2399f956430e8a8fe18806ff2b398a3a"
server_key_version: "kuduclusterkey@0"
I20260812 03:10:43.531772 8553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance:
uuid: "1a61de979223434992ead55ddce3c027"
format_stamp: "Formatted at 2026-08-12 03:10:43 on dist-test-slave-zpfg"
server_key: "6e8959c1e58e3de30ed699b57f214b67"
server_key_iv: "2399f956430e8a8fe18806ff2b398a3a"
server_key_version: "kuduclusterkey@0"
I20260812 03:10:43.541941 8553 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.006s sys 0.000s
I20260812 03:10:43.574980 8568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:43.583446 8553 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.003s sys 0.000s
I20260812 03:10:43.583665 8553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "1a61de979223434992ead55ddce3c027"
format_stamp: "Formatted at 2026-08-12 03:10:43 on dist-test-slave-zpfg"
server_key: "6e8959c1e58e3de30ed699b57f214b67"
server_key_iv: "2399f956430e8a8fe18806ff2b398a3a"
server_key_version: "kuduclusterkey@0"
I20260812 03:10:43.583854 8553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:10:43.639065 8553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:10:43.642529 8553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:10:43.642776 8553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:10:43.643563 8553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:10:43.644569 8553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:10:43.644645 8553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:43.644807 8553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:10:43.644865 8553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:43.657511 8553 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:42111
I20260812 03:10:43.657612 8687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:42111 every 8 connection(s)
I20260812 03:10:43.658999 8553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:10:43.666805 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 8553
I20260812 03:10:43.666952 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance
I20260812 03:10:43.667327 27626 external_mini_cluster.cc:1428] Setting key 44a373ebcfa417c924fcb39f550b614d
Aug 12 03:10:43 dist-test-slave-zpfg krb5kdc[7684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504243, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:10:43.680449 8407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:43.663379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:60453 (local address 127.26.250.190:39327)
0812 03:10:43.663518 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:43.663522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:43.664941 (+ 1419us) server_negotiation.cc:407] Connection header received
0812 03:10:43.666437 (+ 1496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:43.666441 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:43.666504 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:43.666596 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:43.669645 (+ 3049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:43.670489 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:43.672309 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:43.672663 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:43.672799 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:43.675882 (+ 3083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:43.675887 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:43.675890 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:43.675921 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:43.677785 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:43.678488 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:43.678493 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:43.678494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:43.678594 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:43.679038 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:43.679042 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:43.679044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:43.679293 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:43.679400 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:43.680131 (+ 731us) server_negotiation.cc:299] Negotiation successful
0812 03:10:43.680280 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:10:43.682891 8690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:43.662034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39327 (local address 127.26.250.131:60453)
0812 03:10:43.664732 (+ 2698us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:43.664809 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:43.666082 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:43.667986 (+ 1904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:43.667997 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:43.668374 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:10:43.669465 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:43.669486 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:43.670777 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:43.670780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:43.672154 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:43.672166 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:43.672865 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:43.672869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:43.673072 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:43.673818 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:10:43.673840 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:10:43.675653 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:10:43.677926 (+ 2273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:43.677933 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:43.677954 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:43.678373 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:43.678707 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:43.678710 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:43.678712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:43.678908 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:43.679415 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:10:43.679422 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:10:43.679815 (+ 393us) client_negotiation.cc:769] Sending connection context
0812 03:10:43.681018 (+ 1203us) client_negotiation.cc:240] Negotiation successful
0812 03:10:43.681429 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2392,"thread_start_us":193,"threads_started":1}
I20260812 03:10:43.684177 8688 heartbeater.cc:347] Connected to a master server at 127.26.250.190:39327
I20260812 03:10:43.684464 8688 heartbeater.cc:464] Registering TS with master...
I20260812 03:10:43.685139 8688 heartbeater.cc:511] Master 127.26.250.190:39327 requested a full tablet report, sending...
I20260812 03:10:43.686451 8182 ts_manager.cc:194] Registered new tserver with Master: 1a61de979223434992ead55ddce3c027 (127.26.250.131:42111)
I20260812 03:10:43.687258 8182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.131@KRBTEST.COM'} at 127.26.250.131:60453
I20260812 03:10:43.694849 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:10:43.695727 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 8269
I20260812 03:10:43.705503 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 8413
I20260812 03:10:43.716259 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 8553
I20260812 03:10:43.727285 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 8150
I20260812 03:10:43.737305 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:39327
--webserver_interface=127.26.250.190
--webserver_port=43541
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:39327
--ranger_config_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.250.148:43877/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.250.148:43877/kms with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:10:43.857290 8691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:10:43.857658 8691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:10:43.857748 8691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:10:43.862084 8691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:10:43.862210 8691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:10:43.862259 8691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:10:43.862323 8691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:10:43.862386 8691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:10:43.868568 8691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.250.148:43877/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:39327
--ranger_config_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:39327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=43541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:10:43.869781 8691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:10:43.871003 8691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:10:43.880966 8696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:43.880975 8697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:43.881042 8699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:43.881111 8691 server_base.cc:1034] running on GCE node
I20260812 03:10:43.882012 8691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:10:43.883358 8691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:10:43.884542 8691 hybrid_clock.cc:648] HybridClock initialized: now 1786504243884505 us; error 55 us; skew 500 ppm
Aug 12 03:10:43 dist-test-slave-zpfg krb5kdc[7684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504243, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:10:43.887581 8691 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:10:43.888602 8691 webserver.cc:460] Webserver started at http://127.26.250.190:43541/ using document root <none> and password file <none>
I20260812 03:10:43.889199 8691 fs_manager.cc:359] Metadata directory not provided
I20260812 03:10:43.889310 8691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 03:10:43 dist-test-slave-zpfg krb5kdc[7684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504243, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for HTTP/127.26.250.148@KRBTEST.COM
I20260812 03:10:43.944073 8691 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:10:43.946704 8707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:43.948251 8691 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:10:43.948390 8691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "dc9655f1863e4ed79f613d57291d889e"
format_stamp: "Formatted at 2026-08-12 03:10:41 on dist-test-slave-zpfg"
server_key: "b228b9b9540b2e602fcd1ae96b00e7b5"
server_key_iv: "37f3af53ac735090abdf7345f3a8b3ed"
server_key_version: "kuduclusterkey@0"
I20260812 03:10:43.948997 8691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:10:43.965807 8691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:10:43.969410 8691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:10:43.969596 8691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:10:43.978526 8691 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:39327
I20260812 03:10:43.978547 8762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:39327 every 8 connection(s)
I20260812 03:10:43.979751 8691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:10:43.984751 8763 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:10:43.986726 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 8691
I20260812 03:10:43.986764 8763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e: Bootstrap starting.
I20260812 03:10:43.987247 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.250.148:43877/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:33039
--local_ip_for_outbound_sockets=127.26.250.129
--tserver_master_addrs=127.26.250.190:39327
--webserver_port=35433
--webserver_interface=127.26.250.129
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.250.148:43877/kms with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:10:44.012463 8763 log.cc:824] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:10:44.022135 8763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:10:44.022715 8763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e: Bootstrap complete.
I20260812 03:10:44.029160 8763 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9655f1863e4ed79f613d57291d889e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39327 } }
I20260812 03:10:44.029750 8763 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc9655f1863e4ed79f613d57291d889e, State: Initialized, Role: FOLLOWER
I20260812 03:10:44.030411 8763 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9655f1863e4ed79f613d57291d889e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39327 } }
I20260812 03:10:44.030579 8763 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:10:44.030659 8763 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:10:44.030779 8763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:10:44.033584 8763 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9655f1863e4ed79f613d57291d889e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39327 } }
I20260812 03:10:44.034020 8763 leader_election.cc:302] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc9655f1863e4ed79f613d57291d889e; no voters:
I20260812 03:10:44.034479 8763 leader_election.cc:288] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 03:10:44.034652 8767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:10:44.034988 8767 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [term 2 LEADER]: Becoming Leader. State: Replica: dc9655f1863e4ed79f613d57291d889e, State: Running, Role: LEADER
I20260812 03:10:44.035398 8767 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9655f1863e4ed79f613d57291d889e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39327 } }
I20260812 03:10:44.035921 8763 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:10:44.036852 8769 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc9655f1863e4ed79f613d57291d889e. Latest consensus state: current_term: 2 leader_uuid: "dc9655f1863e4ed79f613d57291d889e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9655f1863e4ed79f613d57291d889e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39327 } } }
I20260812 03:10:44.036989 8769 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [sys.catalog]: This master's current role is: LEADER
I20260812 03:10:44.037724 8768 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "dc9655f1863e4ed79f613d57291d889e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc9655f1863e4ed79f613d57291d889e" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39327 } } }
I20260812 03:10:44.037793 8775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:10:44.037817 8768 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e [sys.catalog]: This master's current role is: LEADER
I20260812 03:10:44.041262 8775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:10:44.041878 8763 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 03:10:44.042171 8775 catalog_manager.cc:1259] Loaded cluster ID: bbaee1e38b29445e91fa90eb40d96e4b
I20260812 03:10:44.042274 8775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:10:44.044354 8775 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:10:44.044971 8775 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e: Loaded TSK: 0
I20260812 03:10:44.045984 8775 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:10:44.156639 8765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:10:44.157125 8765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:10:44.157296 8765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:10:44.162432 8765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:10:44.162529 8765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:10:44.162642 8765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:10:44.168851 8765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.250.148:43877/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:33039
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=35433
--tserver_master_addrs=127.26.250.190:39327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:10:44.170080 8765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:10:44.171237 8765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:10:44.183516 8802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:44.184199 8805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:44.184706 8803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:44.186295 8765 server_base.cc:1034] running on GCE node
I20260812 03:10:44.186939 8765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:10:44.188007 8765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:10:44.189347 8765 hybrid_clock.cc:648] HybridClock initialized: now 1786504244189304 us; error 67 us; skew 500 ppm
Aug 12 03:10:44 dist-test-slave-zpfg krb5kdc[7684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504244, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:10:44.194610 8765 init.cc:377] Logged in from keytab as kudu/127.26.250.129@KRBTEST.COM (short username kudu)
I20260812 03:10:44.196205 8765 webserver.cc:460] Webserver started at http://127.26.250.129:35433/ using document root <none> and password file <none>
I20260812 03:10:44.197225 8765 fs_manager.cc:359] Metadata directory not provided
I20260812 03:10:44.197374 8765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 03:10:44 dist-test-slave-zpfg krb5kdc[7684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504244, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for HTTP/127.26.250.148@KRBTEST.COM
I20260812 03:10:44.223846 8765 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 03:10:44.229229 8813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:44.231464 8765 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260812 03:10:44.231695 8765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "52363daed5bc43d0b15c2286fe6f1546"
format_stamp: "Formatted at 2026-08-12 03:10:42 on dist-test-slave-zpfg"
server_key: "8c8d5fd1481741cf80ef6303f9e148bb"
server_key_iv: "25e34ac47a2a62c282d3513d26dbfa79"
server_key_version: "kuduclusterkey@0"
I20260812 03:10:44.232532 8765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:10:44.288015 8765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:10:44.292060 8765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:10:44.292343 8765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:10:44.293121 8765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:10:44.294189 8765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:10:44.294301 8765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:44.294445 8765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:10:44.294489 8765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:44.308981 8765 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:33039
I20260812 03:10:44.314834 8765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:10:44.315039 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 8765
I20260812 03:10:44.315611 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.250.148:43877/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:46243
--local_ip_for_outbound_sockets=127.26.250.130
--tserver_master_addrs=127.26.250.190:39327
--webserver_port=36671
--webserver_interface=127.26.250.130
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.250.148:43877/kms with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:10:44.318001 8932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:33039 every 8 connection(s)
Aug 12 03:10:44 dist-test-slave-zpfg krb5kdc[7684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504244, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:10:44.346879 8936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:44.320765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39327 (local address 127.26.250.129:35437)
0812 03:10:44.321375 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:44.321436 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:44.322614 (+ 1178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:44.327564 (+ 4950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:44.327576 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:44.328058 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:10:44.328886 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:44.328913 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:44.331377 (+ 2464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:44.331381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:44.332999 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:44.333015 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:44.334561 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:44.334569 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:44.334881 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:44.335843 (+ 962us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:10:44.335870 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:10:44.338251 (+ 2381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:10:44.341276 (+ 3025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:44.341283 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:44.341301 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:44.341660 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:44.342104 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:44.342108 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:44.342110 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:44.342282 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:44.344567 (+ 2285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:10:44.344574 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:10:44.344961 (+ 387us) client_negotiation.cc:769] Sending connection context
0812 03:10:44.345195 (+ 234us) client_negotiation.cc:240] Negotiation successful
0812 03:10:44.345606 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":183,"threads_started":1}
I20260812 03:10:44.349131 8937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:44.322454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:35437 (local address 127.26.250.190:39327)
0812 03:10:44.325097 (+ 2643us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:44.325125 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:44.325184 (+ 59us) server_negotiation.cc:407] Connection header received
0812 03:10:44.325556 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:44.325575 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:44.326201 (+ 626us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:44.326696 (+ 495us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:44.329207 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:44.331103 (+ 1896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:44.333169 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:44.334039 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:44.334254 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:44.338652 (+ 4398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:44.338661 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:44.338683 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:44.338727 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:44.341099 (+ 2372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:44.341833 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:44.341840 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:44.341845 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:44.341983 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:44.343708 (+ 1725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:44.343712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:44.343714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:44.344340 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:44.344496 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:44.347614 (+ 3118us) server_negotiation.cc:299] Negotiation successful
0812 03:10:44.348014 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2420,"thread_start_us":160,"threads_started":1}
I20260812 03:10:44.352875 8933 heartbeater.cc:347] Connected to a master server at 127.26.250.190:39327
I20260812 03:10:44.353154 8933 heartbeater.cc:464] Registering TS with master...
I20260812 03:10:44.353848 8933 heartbeater.cc:511] Master 127.26.250.190:39327 requested a full tablet report, sending...
W20260812 03:10:44.356594 8933 heartbeater.cc:644] Failed to heartbeat to 127.26.250.190:39327 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 03:10:44.356740 8933 heartbeater.cc:464] Registering TS with master...
I20260812 03:10:44.356963 8933 heartbeater.cc:511] Master 127.26.250.190:39327 requested a full tablet report, sending...
I20260812 03:10:44.358075 8933 heartbeater.cc:464] Registering TS with master...
I20260812 03:10:44.358291 8933 heartbeater.cc:511] Master 127.26.250.190:39327 requested a full tablet report, sending...
W20260812 03:10:44.359319 8933 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:10:44.485419 8935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:10:44.485901 8935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:10:44.486032 8935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:10:44.490602 8935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:10:44.490694 8935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:10:44.490787 8935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:10:44.496827 8935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.250.148:43877/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:46243
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=36671
--tserver_master_addrs=127.26.250.190:39327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:10:44.498248 8935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:10:44.499480 8935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:10:44.514994 8942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:44.515014 8943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:44.515024 8945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:44.516861 8935 server_base.cc:1034] running on GCE node
I20260812 03:10:44.517541 8935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:10:44.518579 8935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:10:44.521919 8935 hybrid_clock.cc:648] HybridClock initialized: now 1786504244521799 us; error 123 us; skew 500 ppm
Aug 12 03:10:44 dist-test-slave-zpfg krb5kdc[7684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504244, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:10:44.526677 8935 init.cc:377] Logged in from keytab as kudu/127.26.250.130@KRBTEST.COM (short username kudu)
I20260812 03:10:44.528288 8935 webserver.cc:460] Webserver started at http://127.26.250.130:36671/ using document root <none> and password file <none>
I20260812 03:10:44.529237 8935 fs_manager.cc:359] Metadata directory not provided
I20260812 03:10:44.529444 8935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 03:10:44 dist-test-slave-zpfg krb5kdc[7684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504244, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for HTTP/127.26.250.148@KRBTEST.COM
I20260812 03:10:44.565636 8935 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:10:44.569473 8954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:44.571450 8935 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 03:10:44.571610 8935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "e3fd0a8e70404793955c3b6f63f05d19"
format_stamp: "Formatted at 2026-08-12 03:10:43 on dist-test-slave-zpfg"
server_key: "677be9fbf72b8d7beba2d13e04419b7e"
server_key_iv: "30f58169b8fc94c366443618a8a9c7f6"
server_key_version: "kuduclusterkey@0"
I20260812 03:10:44.572414 8935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:10:44.612272 8935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:10:44.616356 8935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:10:44.616628 8935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:10:44.617343 8935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:10:44.618247 8935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:10:44.618332 8935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:44.618422 8935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:10:44.618465 8935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:44.630399 8935 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:46243
I20260812 03:10:44.630502 9070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:46243 every 8 connection(s)
I20260812 03:10:44.631734 8935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:10:44.637040 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 8935
I20260812 03:10:44.637631 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.26.250.148:43877/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:42111
--local_ip_for_outbound_sockets=127.26.250.131
--tserver_master_addrs=127.26.250.190:39327
--webserver_port=39523
--webserver_interface=127.26.250.131
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:42341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.26.250.148:43877/kms with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:10:44 dist-test-slave-zpfg krb5kdc[7684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504244, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:10:44.654714 8937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:44.634583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:57789 (local address 127.26.250.190:39327)
0812 03:10:44.636267 (+ 1684us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:44.636273 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:44.642047 (+ 5774us) server_negotiation.cc:407] Connection header received
0812 03:10:44.643526 (+ 1479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:44.643530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:44.643593 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:44.643653 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:44.645334 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:44.645925 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:44.647203 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:44.647472 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:44.647596 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:44.650316 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:44.650320 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:44.650322 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:44.650399 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:44.651998 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:44.652618 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:44.652622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:44.652624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:44.652715 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:44.653237 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:44.653241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:44.653243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:44.653470 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:44.653590 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:44.654386 (+ 796us) server_negotiation.cc:299] Negotiation successful
0812 03:10:44.654543 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1550}
I20260812 03:10:44.656062 9073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:44.635108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39327 (local address 127.26.250.130:57789)
0812 03:10:44.641797 (+ 6689us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:44.641889 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:44.643233 (+ 1344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:44.644144 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:44.644157 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:44.644475 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:10:44.645123 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:44.645146 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:44.646066 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:44.646069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:44.647067 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:44.647077 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:44.647651 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:44.647656 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:44.647852 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:44.648548 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:10:44.648565 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:10:44.650095 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:10:44.652139 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:44.652147 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:44.652166 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:44.652497 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:44.652865 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:44.652868 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:44.652869 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:44.653127 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:44.653630 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:10:44.653640 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:10:44.654004 (+ 364us) client_negotiation.cc:769] Sending connection context
0812 03:10:44.654317 (+ 313us) client_negotiation.cc:240] Negotiation successful
0812 03:10:44.654718 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6378,"thread_start_us":168,"threads_started":1}
I20260812 03:10:44.657524 9071 heartbeater.cc:347] Connected to a master server at 127.26.250.190:39327
I20260812 03:10:44.657876 9071 heartbeater.cc:464] Registering TS with master...
I20260812 03:10:44.658573 9071 heartbeater.cc:511] Master 127.26.250.190:39327 requested a full tablet report, sending...
W20260812 03:10:44.660578 9071 heartbeater.cc:644] Failed to heartbeat to 127.26.250.190:39327 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 03:10:44.660725 9071 heartbeater.cc:464] Registering TS with master...
I20260812 03:10:44.660929 9071 heartbeater.cc:511] Master 127.26.250.190:39327 requested a full tablet report, sending...
I20260812 03:10:44.661896 9071 heartbeater.cc:464] Registering TS with master...
I20260812 03:10:44.662092 9071 heartbeater.cc:511] Master 127.26.250.190:39327 requested a full tablet report, sending...
W20260812 03:10:44.662943 9071 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:10:44.815491 9074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:10:44.816049 9074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:10:44.816192 9074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:10:44.821941 9074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:10:44.822023 9074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:10:44.822131 9074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:10:44.831472 9074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:42341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.26.250.148:43877/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:42111
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=39523
--tserver_master_addrs=127.26.250.190:39327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:10:44.833254 9074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:10:44.834611 9074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:10:44.848273 9080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:44.848294 9079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:44.848860 9082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:44.848832 9074 server_base.cc:1034] running on GCE node
I20260812 03:10:44.849426 9074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:10:44.850610 9074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:10:44.854223 9074 hybrid_clock.cc:648] HybridClock initialized: now 1786504244854186 us; error 61 us; skew 500 ppm
Aug 12 03:10:44 dist-test-slave-zpfg krb5kdc[7684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504244, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:10:44.862426 9074 init.cc:377] Logged in from keytab as kudu/127.26.250.131@KRBTEST.COM (short username kudu)
I20260812 03:10:44.864120 9074 webserver.cc:460] Webserver started at http://127.26.250.131:39523/ using document root <none> and password file <none>
I20260812 03:10:44.864979 9074 fs_manager.cc:359] Metadata directory not provided
I20260812 03:10:44.865146 9074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 03:10:44 dist-test-slave-zpfg krb5kdc[7684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504244, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for HTTP/127.26.250.148@KRBTEST.COM
I20260812 03:10:44.889138 9074 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 03:10:44.892941 9090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:44.895072 9074 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260812 03:10:44.895265 9074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "1a61de979223434992ead55ddce3c027"
format_stamp: "Formatted at 2026-08-12 03:10:43 on dist-test-slave-zpfg"
server_key: "6e8959c1e58e3de30ed699b57f214b67"
server_key_iv: "2399f956430e8a8fe18806ff2b398a3a"
server_key_version: "kuduclusterkey@0"
I20260812 03:10:44.896145 9074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:10:44.913466 9074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:10:44.918330 9074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:10:44.918706 9074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:10:44.919536 9074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:10:44.920608 9074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:10:44.920689 9074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:44.920790 9074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:10:44.920861 9074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:44.935484 9074 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:42111
I20260812 03:10:44.935606 9215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:42111 every 8 connection(s)
I20260812 03:10:44.936875 9074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:10:44.945884 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 9074
Aug 12 03:10:44 dist-test-slave-zpfg krb5kdc[7684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504244, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:10:44.957962 8937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:44.940047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:37829 (local address 127.26.250.190:39327)
0812 03:10:44.940197 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:44.940203 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:44.942946 (+ 2743us) server_negotiation.cc:407] Connection header received
0812 03:10:44.944122 (+ 1176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:44.944126 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:44.944194 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:44.944287 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:44.946448 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:44.947295 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:44.948647 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:44.948996 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:44.949114 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:44.952623 (+ 3509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:44.952628 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:44.952631 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:44.952660 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:44.954840 (+ 2180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:44.955668 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:44.955673 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:44.955678 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:44.955780 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:44.956278 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:44.956282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:44.956284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:44.956561 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:44.956677 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:44.957575 (+ 898us) server_negotiation.cc:299] Negotiation successful
0812 03:10:44.957745 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 03:10:44.959944 9218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:44.940571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39327 (local address 127.26.250.131:37829)
0812 03:10:44.942747 (+ 2176us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:44.942801 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:44.943774 (+ 973us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:44.944563 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:44.944572 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:44.944900 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:10:44.945457 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:44.945472 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:44.947457 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:44.947461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:44.948487 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:44.948499 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:44.949320 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:44.949326 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:44.949579 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:44.950433 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:10:44.950458 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:10:44.952355 (+ 1897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:10:44.955024 (+ 2669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:44.955030 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:44.955053 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:44.955515 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:44.955903 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:44.955907 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:44.955909 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:44.956153 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:44.956744 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:10:44.956757 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:10:44.957191 (+ 434us) client_negotiation.cc:769] Sending connection context
0812 03:10:44.958178 (+ 987us) client_negotiation.cc:240] Negotiation successful
0812 03:10:44.958640 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1947,"thread_start_us":146,"threads_started":1}
I20260812 03:10:44.961105 9220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:44.947146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42552 (local address 127.26.250.190:39327)
0812 03:10:44.948799 (+ 1653us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:44.948816 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:44.949250 (+ 434us) server_negotiation.cc:407] Connection header received
0812 03:10:44.949292 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:44.949294 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:44.949339 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:44.949424 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:44.950719 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:44.951692 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:44.953233 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:44.953746 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:44.953875 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:44.956154 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:44.956159 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:44.956162 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:44.956192 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:44.958708 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:44.959261 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:44.959265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:44.959267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:44.959360 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:44.959686 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:44.959690 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:44.959692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:44.959925 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:44.960140 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:44.960826 (+ 686us) server_negotiation.cc:299] Negotiation successful
0812 03:10:44.960974 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1567,"thread_start_us":129,"threads_started":1}
I20260812 03:10:44.961385 9216 heartbeater.cc:347] Connected to a master server at 127.26.250.190:39327
I20260812 03:10:44.961676 9216 heartbeater.cc:464] Registering TS with master...
I20260812 03:10:44.962390 9216 heartbeater.cc:511] Master 127.26.250.190:39327 requested a full tablet report, sending...
I20260812 03:10:44.971060 8724 ts_manager.cc:194] Registered new tserver with Master: 1a61de979223434992ead55ddce3c027 (127.26.250.131:42111)
I20260812 03:10:44.972685 8724 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.131@KRBTEST.COM'} at 127.26.250.131:37829
Aug 12 03:10:45 dist-test-slave-zpfg krb5kdc[7684](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786504245, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:10:45.359599 8933 heartbeater.cc:464] Registering TS with master...
I20260812 03:10:45.359927 8933 heartbeater.cc:511] Master 127.26.250.190:39327 requested a full tablet report, sending...
I20260812 03:10:45.360754 8724 ts_manager.cc:194] Registered new tserver with Master: 52363daed5bc43d0b15c2286fe6f1546 (127.26.250.129:33039)
I20260812 03:10:45.361505 8724 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.129@KRBTEST.COM'} at 127.26.250.129:35437
I20260812 03:10:45.663214 9071 heartbeater.cc:464] Registering TS with master...
I20260812 03:10:45.663580 9071 heartbeater.cc:511] Master 127.26.250.190:39327 requested a full tablet report, sending...
I20260812 03:10:45.664424 8725 ts_manager.cc:194] Registered new tserver with Master: e3fd0a8e70404793955c3b6f63f05d19 (127.26.250.130:46243)
I20260812 03:10:45.665266 8725 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.130@KRBTEST.COM'} at 127.26.250.130:57789
I20260812 03:10:45.665349 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:10:45.678511 9230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:45.667386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42558 (local address 127.26.250.190:39327)
0812 03:10:45.668392 (+ 1006us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:45.668396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:45.668412 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:10:45.668455 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:45.668461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:45.668513 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:45.669488 (+ 975us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:45.669593 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:45.670419 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:45.671700 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:45.672080 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:45.672608 (+ 528us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:45.673973 (+ 1365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:45.673980 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:45.673984 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:45.674012 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:45.676015 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:45.676572 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:45.676575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:45.676577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:45.676673 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:45.677015 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:45.677019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:45.677020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:45.677264 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:45.677404 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:45.678135 (+ 731us) server_negotiation.cc:299] Negotiation successful
0812 03:10:45.678291 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":929,"thread_start_us":80,"threads_started":1}
I20260812 03:10:45.683271 8725 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42558:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:10:45.687849 8725 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:10:45.708701 9236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:45.700110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:46243 (local address 127.0.0.1:54670)
0812 03:10:45.700981 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:45.701047 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:45.701200 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:45.703242 (+ 2042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:45.703248 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:45.703275 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:10:45.703444 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:45.703451 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:45.705114 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:45.705118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:45.706655 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:45.706668 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:45.708082 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:45.708086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:45.708221 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:45.708267 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 03:10:45.708389 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 03:10:45.708531 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":139,"threads_started":1}
I20260812 03:10:45.709587 9237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:45.700238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54670 (local address 127.26.250.130:46243)
0812 03:10:45.701519 (+ 1281us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:45.701527 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:45.701553 (+ 26us) server_negotiation.cc:407] Connection header received
0812 03:10:45.701766 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:45.701776 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:45.703025 (+ 1249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:45.703611 (+ 586us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:10:45.703783 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:45.704974 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:45.706824 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:45.707708 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:45.707854 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:45.708006 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:45.709307 (+ 1301us) server_negotiation.cc:299] Negotiation successful
0812 03:10:45.709432 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1163,"thread_start_us":232,"threads_started":1}
I20260812 03:10:45.713117 9238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:45.702172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:33039 (local address 127.0.0.1:56578)
0812 03:10:45.706871 (+ 4699us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:45.706886 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:45.706975 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:45.707676 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:45.707679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:45.707697 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:10:45.707796 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:45.707801 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:45.709276 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:45.709279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:45.711348 (+ 2069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:45.711360 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:45.712704 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:45.712708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:45.712812 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:45.712836 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:10:45.712904 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 03:10:45.712976 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4632,"thread_start_us":95,"threads_started":1}
I20260812 03:10:45.713565 9242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:45.702525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56578 (local address 127.26.250.129:33039)
0812 03:10:45.703339 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:45.703346 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:45.707081 (+ 3735us) server_negotiation.cc:407] Connection header received
0812 03:10:45.707167 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:45.707172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:45.707381 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:45.707527 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:10:45.707925 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:45.709127 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:45.711501 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:45.712383 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:45.712506 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:45.712637 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:45.713314 (+ 677us) server_negotiation.cc:299] Negotiation successful
0812 03:10:45.713427 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":70,"threads_started":1}
I20260812 03:10:45.715379 9003 tablet_service.cc:1467] Processing CreateTablet for tablet 3ce3b82a2856438191a6533eaf3a350f (DEFAULT_TABLE table=test-table [id=cccc44e01ced4f419700aa7a2b0e2b36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:10:45.716727 9003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ce3b82a2856438191a6533eaf3a350f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:10:45.719501 8864 tablet_service.cc:1467] Processing CreateTablet for tablet 3ce3b82a2856438191a6533eaf3a350f (DEFAULT_TABLE table=test-table [id=cccc44e01ced4f419700aa7a2b0e2b36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:10:45.721942 8864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ce3b82a2856438191a6533eaf3a350f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:10:45.727914 9243 tablet_bootstrap.cc:492] T 3ce3b82a2856438191a6533eaf3a350f P e3fd0a8e70404793955c3b6f63f05d19: Bootstrap starting.
I20260812 03:10:45.728824 9240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:45.701372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:42111 (local address 127.0.0.1:43604)
0812 03:10:45.702989 (+ 1617us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:45.703005 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:45.703093 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:45.709922 (+ 6829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:45.709926 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:45.709953 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:10:45.710084 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:45.710091 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:45.715637 (+ 5546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:45.715640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:45.716913 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:45.716925 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:45.728302 (+ 11377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:45.728307 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:45.728440 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:45.728470 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 03:10:45.728552 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 03:10:45.728652 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1548,"thread_start_us":112,"threads_started":1}
I20260812 03:10:45.729312 9239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:45.701552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43604 (local address 127.26.250.131:42111)
0812 03:10:45.705290 (+ 3738us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:45.705297 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:45.705321 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:10:45.705397 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:45.705402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:45.706457 (+ 1055us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:45.706611 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:10:45.710250 (+ 3639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:45.711515 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:45.725123 (+ 13608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:45.726117 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:45.726308 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:45.728220 (+ 1912us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:45.729022 (+ 802us) server_negotiation.cc:299] Negotiation successful
0812 03:10:45.729163 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3617,"thread_start_us":151,"threads_started":1}
I20260812 03:10:45.731073 9243 tablet_bootstrap.cc:654] T 3ce3b82a2856438191a6533eaf3a350f P e3fd0a8e70404793955c3b6f63f05d19: Neither blocks nor log segments found. Creating new log.
I20260812 03:10:45.732017 9243 log.cc:824] T 3ce3b82a2856438191a6533eaf3a350f P e3fd0a8e70404793955c3b6f63f05d19: Log is configured to *not* fsync() on all Append() calls
I20260812 03:10:45.732745 9147 tablet_service.cc:1467] Processing CreateTablet for tablet 3ce3b82a2856438191a6533eaf3a350f (DEFAULT_TABLE table=test-table [id=cccc44e01ced4f419700aa7a2b0e2b36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:10:45.734040 9147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ce3b82a2856438191a6533eaf3a350f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:10:45.735639 9243 tablet_bootstrap.cc:492] T 3ce3b82a2856438191a6533eaf3a350f P e3fd0a8e70404793955c3b6f63f05d19: No bootstrap required, opened a new log
I20260812 03:10:45.735877 9243 ts_tablet_manager.cc:1397] T 3ce3b82a2856438191a6533eaf3a350f P e3fd0a8e70404793955c3b6f63f05d19: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260812 03:10:45.740377 9243 raft_consensus.cc:348] T 3ce3b82a2856438191a6533eaf3a350f P e3fd0a8e70404793955c3b6f63f05d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3fd0a8e70404793955c3b6f63f05d19" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46243 } } peers { permanent_uuid: "1a61de979223434992ead55ddce3c027" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 42111 } } peers { permanent_uuid: "52363daed5bc43d0b15c2286fe6f1546" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 33039 } }
I20260812 03:10:45.740666 9243 raft_consensus.cc:374] T 3ce3b82a2856438191a6533eaf3a350f P e3fd0a8e70404793955c3b6f63f05d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:10:45.740718 9243 raft_consensus.cc:729] T 3ce3b82a2856438191a6533eaf3a350f P e3fd0a8e70404793955c3b6f63f05d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3fd0a8e70404793955c3b6f63f05d19, State: Initialized, Role: FOLLOWER
I20260812 03:10:45.741230 9243 consensus_queue.cc:260] T 3ce3b82a2856438191a6533eaf3a350f P e3fd0a8e70404793955c3b6f63f05d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3fd0a8e70404793955c3b6f63f05d19" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46243 } } peers { permanent_uuid: "1a61de979223434992ead55ddce3c027" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 42111 } } peers { permanent_uuid: "52363daed5bc43d0b15c2286fe6f1546" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 33039 } }
I20260812 03:10:45.742287 9243 ts_tablet_manager.cc:1428] T 3ce3b82a2856438191a6533eaf3a350f P e3fd0a8e70404793955c3b6f63f05d19: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 03:10:45.742924 9244 tablet_bootstrap.cc:492] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546: Bootstrap starting.
I20260812 03:10:45.744522 9071 heartbeater.cc:503] Master 127.26.250.190:39327 was elected leader, sending a full tablet report...
I20260812 03:10:45.745540 9244 tablet_bootstrap.cc:654] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546: Neither blocks nor log segments found. Creating new log.
I20260812 03:10:45.746556 9244 log.cc:824] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546: Log is configured to *not* fsync() on all Append() calls
I20260812 03:10:45.749670 9244 tablet_bootstrap.cc:492] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546: No bootstrap required, opened a new log
I20260812 03:10:45.749907 9244 ts_tablet_manager.cc:1397] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260812 03:10:45.750718 9247 tablet_bootstrap.cc:492] T 3ce3b82a2856438191a6533eaf3a350f P 1a61de979223434992ead55ddce3c027: Bootstrap starting.
I20260812 03:10:45.753268 9247 tablet_bootstrap.cc:654] T 3ce3b82a2856438191a6533eaf3a350f P 1a61de979223434992ead55ddce3c027: Neither blocks nor log segments found. Creating new log.
I20260812 03:10:45.754431 9247 log.cc:824] T 3ce3b82a2856438191a6533eaf3a350f P 1a61de979223434992ead55ddce3c027: Log is configured to *not* fsync() on all Append() calls
I20260812 03:10:45.754736 9244 raft_consensus.cc:348] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3fd0a8e70404793955c3b6f63f05d19" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46243 } } peers { permanent_uuid: "1a61de979223434992ead55ddce3c027" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 42111 } } peers { permanent_uuid: "52363daed5bc43d0b15c2286fe6f1546" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 33039 } }
I20260812 03:10:45.755139 9244 raft_consensus.cc:374] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:10:45.755245 9244 raft_consensus.cc:729] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52363daed5bc43d0b15c2286fe6f1546, State: Initialized, Role: FOLLOWER
I20260812 03:10:45.755868 9244 consensus_queue.cc:260] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3fd0a8e70404793955c3b6f63f05d19" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46243 } } peers { permanent_uuid: "1a61de979223434992ead55ddce3c027" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 42111 } } peers { permanent_uuid: "52363daed5bc43d0b15c2286fe6f1546" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 33039 } }
I20260812 03:10:45.756913 9247 tablet_bootstrap.cc:492] T 3ce3b82a2856438191a6533eaf3a350f P 1a61de979223434992ead55ddce3c027: No bootstrap required, opened a new log
I20260812 03:10:45.757124 9247 ts_tablet_manager.cc:1397] T 3ce3b82a2856438191a6533eaf3a350f P 1a61de979223434992ead55ddce3c027: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260812 03:10:45.757298 9244 ts_tablet_manager.cc:1428] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 03:10:45.757706 8933 heartbeater.cc:503] Master 127.26.250.190:39327 was elected leader, sending a full tablet report...
I20260812 03:10:45.762280 9247 raft_consensus.cc:348] T 3ce3b82a2856438191a6533eaf3a350f P 1a61de979223434992ead55ddce3c027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3fd0a8e70404793955c3b6f63f05d19" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46243 } } peers { permanent_uuid: "1a61de979223434992ead55ddce3c027" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 42111 } } peers { permanent_uuid: "52363daed5bc43d0b15c2286fe6f1546" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 33039 } }
I20260812 03:10:45.762629 9247 raft_consensus.cc:374] T 3ce3b82a2856438191a6533eaf3a350f P 1a61de979223434992ead55ddce3c027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:10:45.762784 9247 raft_consensus.cc:729] T 3ce3b82a2856438191a6533eaf3a350f P 1a61de979223434992ead55ddce3c027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a61de979223434992ead55ddce3c027, State: Initialized, Role: FOLLOWER
I20260812 03:10:45.763381 9247 consensus_queue.cc:260] T 3ce3b82a2856438191a6533eaf3a350f P 1a61de979223434992ead55ddce3c027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3fd0a8e70404793955c3b6f63f05d19" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46243 } } peers { permanent_uuid: "1a61de979223434992ead55ddce3c027" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 42111 } } peers { permanent_uuid: "52363daed5bc43d0b15c2286fe6f1546" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 33039 } }
I20260812 03:10:45.764289 9216 heartbeater.cc:503] Master 127.26.250.190:39327 was elected leader, sending a full tablet report...
I20260812 03:10:45.764719 9247 ts_tablet_manager.cc:1428] T 3ce3b82a2856438191a6533eaf3a350f P 1a61de979223434992ead55ddce3c027: Time spent starting tablet: real 0.008s user 0.007s sys 0.001s
Aug 12 03:10:45 dist-test-slave-zpfg krb5kdc[7684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504245, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for HTTP/127.26.250.148@KRBTEST.COM
W20260812 03:10:45.820148 8934 tablet.cc:2369] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:10:45.826177 9250 raft_consensus.cc:482] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:10:45.826519 9250 raft_consensus.cc:504] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3fd0a8e70404793955c3b6f63f05d19" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46243 } } peers { permanent_uuid: "1a61de979223434992ead55ddce3c027" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 42111 } } peers { permanent_uuid: "52363daed5bc43d0b15c2286fe6f1546" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 33039 } }
I20260812 03:10:45.827946 9250 leader_election.cc:288] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3fd0a8e70404793955c3b6f63f05d19 (127.26.250.130:46243), 1a61de979223434992ead55ddce3c027 (127.26.250.131:42111)
I20260812 03:10:45.833849 9239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:45.828717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:59941 (local address 127.26.250.131:42111)
0812 03:10:45.828846 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:45.828851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:45.829293 (+ 442us) server_negotiation.cc:407] Connection header received
0812 03:10:45.829485 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:45.829488 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:45.829539 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:45.829586 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:10:45.830005 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:45.830821 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:45.832410 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:45.833052 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:45.833148 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:45.833196 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:45.833614 (+ 418us) server_negotiation.cc:299] Negotiation successful
0812 03:10:45.833711 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 03:10:45.833850 9237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:45.828703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:47057 (local address 127.26.250.130:46243)
0812 03:10:45.828848 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:45.828852 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:45.829030 (+ 178us) server_negotiation.cc:407] Connection header received
0812 03:10:45.829260 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:45.829264 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:45.829319 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:45.829370 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:10:45.829828 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:45.830654 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:45.832363 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:45.832963 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:45.833075 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:45.833134 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:45.833630 (+ 496us) server_negotiation.cc:299] Negotiation successful
0812 03:10:45.833712 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:10:45.834381 9253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:45.828255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:46243 (local address 127.26.250.129:47057)
0812 03:10:45.828930 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:45.828947 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:45.829078 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:45.829516 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:45.829519 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:45.829537 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:10:45.829685 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:45.829691 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:45.830815 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:45.830818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:45.832199 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:45.832211 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:45.833313 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:45.833319 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:45.833473 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:45.833508 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 03:10:45.834079 (+ 571us) client_negotiation.cc:240] Negotiation successful
0812 03:10:45.834189 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":3,"thread_start_us":120,"threads_started":1}
I20260812 03:10:45.834709 9252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:45.828291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:42111 (local address 127.26.250.129:59941)
0812 03:10:45.829202 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:45.829215 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:45.829368 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:45.829740 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:45.829743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:45.829759 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:10:45.829881 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:45.829886 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:45.830966 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:45.830969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:45.832257 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:45.832268 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:45.833296 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:45.833299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:45.833411 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:45.833497 (+ 86us) client_negotiation.cc:769] Sending connection context
0812 03:10:45.834330 (+ 833us) client_negotiation.cc:240] Negotiation successful
0812 03:10:45.834435 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"mutex_wait_us":66,"thread_start_us":126,"threads_started":1}
I20260812 03:10:45.835391 9024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ce3b82a2856438191a6533eaf3a350f" candidate_uuid: "52363daed5bc43d0b15c2286fe6f1546" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3fd0a8e70404793955c3b6f63f05d19" is_pre_election: true
I20260812 03:10:45.835491 9159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ce3b82a2856438191a6533eaf3a350f" candidate_uuid: "52363daed5bc43d0b15c2286fe6f1546" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a61de979223434992ead55ddce3c027" is_pre_election: true
I20260812 03:10:45.835745 9024 raft_consensus.cc:2454] T 3ce3b82a2856438191a6533eaf3a350f P e3fd0a8e70404793955c3b6f63f05d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52363daed5bc43d0b15c2286fe6f1546 in term 0.
I20260812 03:10:45.835861 9159 raft_consensus.cc:2454] T 3ce3b82a2856438191a6533eaf3a350f P 1a61de979223434992ead55ddce3c027 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52363daed5bc43d0b15c2286fe6f1546 in term 0.
I20260812 03:10:45.836429 8816 leader_election.cc:302] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52363daed5bc43d0b15c2286fe6f1546, e3fd0a8e70404793955c3b6f63f05d19; no voters:
I20260812 03:10:45.836776 9250 raft_consensus.cc:2781] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:10:45.836858 9250 raft_consensus.cc:482] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:10:45.836917 9250 raft_consensus.cc:3037] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:10:45.838411 9250 raft_consensus.cc:504] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3fd0a8e70404793955c3b6f63f05d19" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46243 } } peers { permanent_uuid: "1a61de979223434992ead55ddce3c027" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 42111 } } peers { permanent_uuid: "52363daed5bc43d0b15c2286fe6f1546" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 33039 } }
I20260812 03:10:45.838868 9250 leader_election.cc:288] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546 [CANDIDATE]: Term 1 election: Requested vote from peers e3fd0a8e70404793955c3b6f63f05d19 (127.26.250.130:46243), 1a61de979223434992ead55ddce3c027 (127.26.250.131:42111)
I20260812 03:10:45.839365 9159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ce3b82a2856438191a6533eaf3a350f" candidate_uuid: "52363daed5bc43d0b15c2286fe6f1546" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a61de979223434992ead55ddce3c027"
I20260812 03:10:45.839365 9024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ce3b82a2856438191a6533eaf3a350f" candidate_uuid: "52363daed5bc43d0b15c2286fe6f1546" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3fd0a8e70404793955c3b6f63f05d19"
I20260812 03:10:45.839498 9159 raft_consensus.cc:3037] T 3ce3b82a2856438191a6533eaf3a350f P 1a61de979223434992ead55ddce3c027 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:10:45.839514 9024 raft_consensus.cc:3037] T 3ce3b82a2856438191a6533eaf3a350f P e3fd0a8e70404793955c3b6f63f05d19 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:10:45.840911 9159 raft_consensus.cc:2454] T 3ce3b82a2856438191a6533eaf3a350f P 1a61de979223434992ead55ddce3c027 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52363daed5bc43d0b15c2286fe6f1546 in term 1.
I20260812 03:10:45.841025 9024 raft_consensus.cc:2454] T 3ce3b82a2856438191a6533eaf3a350f P e3fd0a8e70404793955c3b6f63f05d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52363daed5bc43d0b15c2286fe6f1546 in term 1.
I20260812 03:10:45.841343 8814 leader_election.cc:302] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a61de979223434992ead55ddce3c027, 52363daed5bc43d0b15c2286fe6f1546; no voters:
I20260812 03:10:45.841648 9250 raft_consensus.cc:2781] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:10:45.841912 9250 raft_consensus.cc:686] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546 [term 1 LEADER]: Becoming Leader. State: Replica: 52363daed5bc43d0b15c2286fe6f1546, State: Running, Role: LEADER
I20260812 03:10:45.842324 9250 consensus_queue.cc:237] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3fd0a8e70404793955c3b6f63f05d19" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46243 } } peers { permanent_uuid: "1a61de979223434992ead55ddce3c027" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 42111 } } peers { permanent_uuid: "52363daed5bc43d0b15c2286fe6f1546" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 33039 } }
I20260812 03:10:45.846629 8724 catalog_manager.cc:5495] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52363daed5bc43d0b15c2286fe6f1546 (127.26.250.129). New cstate: current_term: 1 leader_uuid: "52363daed5bc43d0b15c2286fe6f1546" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3fd0a8e70404793955c3b6f63f05d19" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a61de979223434992ead55ddce3c027" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 42111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52363daed5bc43d0b15c2286fe6f1546" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 33039 } health_report { overall_health: HEALTHY } } }
I20260812 03:10:45.872023 9242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:45.866850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56582 (local address 127.26.250.129:33039)
0812 03:10:45.867013 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:45.867017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:45.867034 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:10:45.867241 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:45.867245 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:45.867330 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:45.867467 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:10:45.867811 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:45.868648 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:45.870023 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:45.870416 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:45.870525 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:45.870902 (+ 377us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:10:45.871525 (+ 623us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:10:45.871651 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:45.871772 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 03:10:45.871842 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 03:10:45.881807 9159 raft_consensus.cc:1261] T 3ce3b82a2856438191a6533eaf3a350f P 1a61de979223434992ead55ddce3c027 [term 1 FOLLOWER]: Refusing update from remote peer 52363daed5bc43d0b15c2286fe6f1546: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:10:45.881837 9024 raft_consensus.cc:1261] T 3ce3b82a2856438191a6533eaf3a350f P e3fd0a8e70404793955c3b6f63f05d19 [term 1 FOLLOWER]: Refusing update from remote peer 52363daed5bc43d0b15c2286fe6f1546: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:10:45.882740 9250 consensus_queue.cc:1035] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3fd0a8e70404793955c3b6f63f05d19" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:10:45.883179 9254 consensus_queue.cc:1035] T 3ce3b82a2856438191a6533eaf3a350f P 52363daed5bc43d0b15c2286fe6f1546 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a61de979223434992ead55ddce3c027" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 42111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 03:10:45.885253 9072 tablet.cc:2369] T 3ce3b82a2856438191a6533eaf3a350f P e3fd0a8e70404793955c3b6f63f05d19: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:10:45.900970 9258 mvcc.cc:205] Tried to move back new op lower bound from 7317521391123107840 to 7317521390973870080. Current Snapshot: MvccSnapshot[applied={T|T < 7317521391123107840}]
I20260812 03:10:45.901486 9257 mvcc.cc:205] Tried to move back new op lower bound from 7317521391123107840 to 7317521390973870080. Current Snapshot: MvccSnapshot[applied={T|T < 7317521391123107840}]
I20260812 03:10:45.905006 9259 mvcc.cc:205] Tried to move back new op lower bound from 7317521391123107840 to 7317521390973870080. Current Snapshot: MvccSnapshot[applied={T|T < 7317521391123107840}]
I20260812 03:10:45.915131 8725 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:10:45.915342 8725 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:10:45.918462 8725 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dc9655f1863e4ed79f613d57291d889e: Sending DeleteTablet for 3 replicas of tablet 3ce3b82a2856438191a6533eaf3a350f
I20260812 03:10:45.920148 9003 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ce3b82a2856438191a6533eaf3a350f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:10:45 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:54670
I20260812 03:10:45.920807 8864 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ce3b82a2856438191a6533eaf3a350f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:10:45 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:56578
I20260812 03:10:45.921347 9147 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ce3b82a2856438191a6533eaf3a350f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:10:45 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:43604
I20260812 03:10:45.923050 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 8765
I20260812 03:10:45.926985 9266 tablet_replica.cc:330] T 3ce3b82a2856438191a6533eaf3a350f P e3fd0a8e70404793955c3b6f63f05d19: stopping tablet replica
I20260812 03:10:45.927454 9266 raft_consensus.cc:2229] T 3ce3b82a2856438191a6533eaf3a350f P e3fd0a8e70404793955c3b6f63f05d19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:10:45.927933 9266 raft_consensus.cc:2258] T 3ce3b82a2856438191a6533eaf3a350f P e3fd0a8e70404793955c3b6f63f05d19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:10:45.928438 9268 tablet_replica.cc:330] T 3ce3b82a2856438191a6533eaf3a350f P 1a61de979223434992ead55ddce3c027: stopping tablet replica
I20260812 03:10:45.928874 9268 raft_consensus.cc:2229] T 3ce3b82a2856438191a6533eaf3a350f P 1a61de979223434992ead55ddce3c027 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:10:45.930091 9266 ts_tablet_manager.cc:1905] T 3ce3b82a2856438191a6533eaf3a350f P e3fd0a8e70404793955c3b6f63f05d19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:10:45.930554 9268 raft_consensus.cc:2258] T 3ce3b82a2856438191a6533eaf3a350f P 1a61de979223434992ead55ddce3c027 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:10:45.933812 9266 ts_tablet_manager.cc:1918] T 3ce3b82a2856438191a6533eaf3a350f P e3fd0a8e70404793955c3b6f63f05d19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:10:45.934120 9266 log.cc:1196] T 3ce3b82a2856438191a6533eaf3a350f P e3fd0a8e70404793955c3b6f63f05d19: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-1/wal/wals/3ce3b82a2856438191a6533eaf3a350f
I20260812 03:10:45.935407 9268 ts_tablet_manager.cc:1905] T 3ce3b82a2856438191a6533eaf3a350f P 1a61de979223434992ead55ddce3c027: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 03:10:45.936272 8710 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.250.129:33039 (error 108)
I20260812 03:10:45.936429 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 8935
I20260812 03:10:45.941257 9268 ts_tablet_manager.cc:1918] T 3ce3b82a2856438191a6533eaf3a350f P 1a61de979223434992ead55ddce3c027: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:10:45.948417 9268 log.cc:1196] T 3ce3b82a2856438191a6533eaf3a350f P 1a61de979223434992ead55ddce3c027: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/3ce3b82a2856438191a6533eaf3a350f
I20260812 03:10:45.948978 9268 ts_tablet_manager.cc:1939] T 3ce3b82a2856438191a6533eaf3a350f P 1a61de979223434992ead55ddce3c027: Deleting consensus metadata
I20260812 03:10:45.950692 8708 catalog_manager.cc:4841] TS 1a61de979223434992ead55ddce3c027 (127.26.250.131:42111): tablet 3ce3b82a2856438191a6533eaf3a350f (table test-table [id=cccc44e01ced4f419700aa7a2b0e2b36]) successfully deleted
I20260812 03:10:45.951850 9240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:45.951494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:33039 (local address 127.0.0.1:56596)
0812 03:10:45.951664 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:45.951755 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.129:33039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 03:10:45.952049 8710 catalog_manager.cc:4584] TS 52363daed5bc43d0b15c2286fe6f1546 (127.26.250.129:33039): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3ce3b82a2856438191a6533eaf3a350f: Network error: Client connection negotiation failed: client connection to 127.26.250.129:33039: connect: Connection refused (error 111)
W20260812 03:10:45.952829 9240 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:10:45.952512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:46243 (local address 127.0.0.1:54676)
0812 03:10:45.952638 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:45.952656 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:45.952741 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.130:46243: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 03:10:45.953189 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 9074
I20260812 03:10:45.964007 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 8691
I20260812 03:10:45.973495 27626 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 03:10:46.319284 27626 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 03:10:46.319410 27626 mini_ranger.cc:66] Stopping Ranger...
I20260812 03:10:46.675501 27626 mini_ranger.cc:68] Stopped Ranger
2026-08-12 03:10:46.675 UTC [7716] LOG: received smart shutdown request
2026-08-12 03:10:46.680 UTC [7716] LOG: background worker "logical replication launcher" (PID 7725) exited with exit code 1
2026-08-12 03:10:46.680 UTC [7720] LOG: shutting down
2026-08-12 03:10:46.879 UTC [7716] LOG: database system is shut down
2026-08-12T03:10:46Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53237 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:10:47 dist-test-slave-zpfg krb5kdc[9279](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:10:47 dist-test-slave-zpfg krb5kdc[9279](info): set up 2 sockets
Aug 12 03:10:47 dist-test-slave-zpfg krb5kdc[9279](info): commencing operation
krb5kdc: starting...
W20260812 03:10:48.096091 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:10:48 dist-test-slave-zpfg krb5kdc[9279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:10:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:10:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:10:48.238041 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:42369
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:44175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:42369,127.26.250.189:33581,127.26.250.188:39301
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:10:48.359081 9292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:10:48.359452 9292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:10:48.359539 9292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:10:48.363724 9292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:10:48.363839 9292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:10:48.363878 9292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:10:48.363915 9292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:10:48.363951 9292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:10:48.369977 9292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:44175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:42369,127.26.250.189:33581,127.26.250.188:39301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:42369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:10:48.371275 9292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:10:48.372534 9292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:10:48.381639 9298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:48.381569 9297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:48.381639 9300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:48.381924 9292 server_base.cc:1034] running on GCE node
I20260812 03:10:48.382772 9292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:10:48.384083 9292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:10:48.385454 9292 hybrid_clock.cc:648] HybridClock initialized: now 1786504248385416 us; error 63 us; skew 500 ppm
Aug 12 03:10:48 dist-test-slave-zpfg krb5kdc[9279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504248, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:10:48.388567 9292 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:10:48.389523 9292 webserver.cc:460] Webserver started at http://127.26.250.190:34531/ using document root <none> and password file <none>
I20260812 03:10:48.390121 9292 fs_manager.cc:359] Metadata directory not provided
I20260812 03:10:48.390224 9292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:10:48.390537 9292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:10:48.392733 9292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "72361de1616e4b56b5e96a3d7933bb22"
format_stamp: "Formatted at 2026-08-12 03:10:48 on dist-test-slave-zpfg"
server_key: "1f8b0765144cba4657d0a0b6cde61b1e"
server_key_iv: "854dadc78008daee88b86fdbe665e057"
server_key_version: "encryptionkey@0"
I20260812 03:10:48.393318 9292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "72361de1616e4b56b5e96a3d7933bb22"
format_stamp: "Formatted at 2026-08-12 03:10:48 on dist-test-slave-zpfg"
server_key: "1f8b0765144cba4657d0a0b6cde61b1e"
server_key_iv: "854dadc78008daee88b86fdbe665e057"
server_key_version: "encryptionkey@0"
I20260812 03:10:48.397389 9292 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:10:48.400173 9307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:48.401523 9292 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:10:48.401678 9292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "72361de1616e4b56b5e96a3d7933bb22"
format_stamp: "Formatted at 2026-08-12 03:10:48 on dist-test-slave-zpfg"
server_key: "1f8b0765144cba4657d0a0b6cde61b1e"
server_key_iv: "854dadc78008daee88b86fdbe665e057"
server_key_version: "encryptionkey@0"
I20260812 03:10:48.401804 9292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:10:48.417619 9292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:10:48.421042 9292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:10:48.421259 9292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:10:48.428938 9292 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:42369
I20260812 03:10:48.429286 9361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:42369 every 8 connection(s)
I20260812 03:10:48.430040 9292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:10:48.433318 9362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:10:48.436676 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 9292
I20260812 03:10:48.436846 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:10:48.437141 27626 external_mini_cluster.cc:1428] Setting key 35a12d4f3e66906c7dfa8a9ce7cc3134
I20260812 03:10:48.437799 9362 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42369 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.250.189@KRBTEST.COM; defaulting to no policy
Aug 12 03:10:48 dist-test-slave-zpfg krb5kdc[9279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504248, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.189@KRBTEST.COM" created.
I20260812 03:10:48.455758 9365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.440404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:42369 (local address 127.0.0.1:50736)
0812 03:10:48.440937 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.440992 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:48.441954 (+ 962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:48.442846 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:48.442851 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:48.443326 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:10:48.443663 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:48.443679 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.444915 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.444918 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:48.445723 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:48.445738 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.446828 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.446831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:48.446932 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:48.447678 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:10:48.447696 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:10:48.449429 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:10:48.451901 (+ 2472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:48.451915 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:48.451928 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:48.452291 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:48.452668 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:48.452674 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:48.452679 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:48.452871 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:48.453572 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:10:48.453583 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:10:48.453804 (+ 221us) client_negotiation.cc:769] Sending connection context
0812 03:10:48.454024 (+ 220us) client_negotiation.cc:240] Negotiation successful
0812 03:10:48.454638 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":197,"spinlock_wait_cycles":896,"thread_start_us":164,"threads_started":1}
I20260812 03:10:48.455760 9366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.440405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50736 (local address 127.26.250.190:42369)
0812 03:10:48.440892 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:48.440907 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:48.441112 (+ 205us) server_negotiation.cc:407] Connection header received
0812 03:10:48.442291 (+ 1179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:48.442299 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:48.442590 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:48.442842 (+ 252us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:48.443842 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.444764 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.445872 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.446627 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.446832 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:48.449617 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:48.449623 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:48.449635 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:48.449657 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:48.451675 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:48.452419 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:48.452424 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:48.452426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:48.452547 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:48.452976 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:48.452978 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:48.452979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:48.453432 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:48.453535 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:48.454085 (+ 550us) server_negotiation.cc:299] Negotiation successful
0812 03:10:48.454496 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":345,"spinlock_wait_cycles":18688,"thread_start_us":184,"threads_started":1}
I20260812 03:10:48.458532 9362 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.189" port: 33581 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:10:48.459583 9365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.459123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:33581 (local address 127.0.0.1:59118)
0812 03:10:48.459436 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.459515 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.189:33581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220}
W20260812 03:10:48.459910 9311 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.250.189:33581: connect: Connection refused (error 111)
I20260812 03:10:48.461127 9365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.460883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:33581 (local address 127.0.0.1:59126)
0812 03:10:48.461010 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.461070 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.189:33581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 03:10:48.461400 9362 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.189:33581: Network error: Client connection negotiation failed: client connection to 127.26.250.189:33581: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.189.keytab.
Entry for principal kudu/127.26.250.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.189.keytab.
I20260812 03:10:48.470459 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.189:33581
--webserver_interface=127.26.250.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.189.keytab
--principal=kudu/127.26.250.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:44175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:42369,127.26.250.189:33581,127.26.250.188:39301
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:10:48.510697 9362 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.189" port: 33581 } attempt: 1
I20260812 03:10:48.511651 9365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.511320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:33581 (local address 127.0.0.1:59130)
0812 03:10:48.511493 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.511563 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.189:33581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260812 03:10:48.512737 9365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.512505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:33581 (local address 127.0.0.1:59132)
0812 03:10:48.512628 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.512678 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.189:33581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 03:10:48.512984 9362 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.189:33581: Network error: Client connection negotiation failed: client connection to 127.26.250.189:33581: connect: Connection refused (error 111)
I20260812 03:10:48.581192 9362 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.189" port: 33581 } attempt: 2
I20260812 03:10:48.582325 9365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.581966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:33581 (local address 127.0.0.1:59140)
0812 03:10:48.582134 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.582205 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.189:33581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260812 03:10:48.583321 9365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.583058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:33581 (local address 127.0.0.1:59156)
0812 03:10:48.583171 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.583222 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.189:33581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260812 03:10:48.583475 9362 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.189:33581: Network error: Client connection negotiation failed: client connection to 127.26.250.189:33581: connect: Connection refused (error 111)
W20260812 03:10:48.591979 9369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:10:48.592299 9369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:10:48.592445 9369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:10:48.596662 9369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:10:48.596765 9369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:10:48.596804 9369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:10:48.596863 9369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:10:48.596903 9369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:10:48.603022 9369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:44175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:42369,127.26.250.189:33581,127.26.250.188:39301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.189.keytab
--principal=kudu/127.26.250.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.189:33581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.250.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:10:48.604303 9369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:10:48.605634 9369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:10:48.615610 9375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:48.615623 9377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:48.615612 9374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:48.615852 9369 server_base.cc:1034] running on GCE node
I20260812 03:10:48.616911 9369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:10:48.618170 9369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:10:48.619398 9369 hybrid_clock.cc:648] HybridClock initialized: now 1786504248619383 us; error 32 us; skew 500 ppm
Aug 12 03:10:48 dist-test-slave-zpfg krb5kdc[9279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504248, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:10:48.622637 9369 init.cc:377] Logged in from keytab as kudu/127.26.250.189@KRBTEST.COM (short username kudu)
I20260812 03:10:48.623637 9369 webserver.cc:460] Webserver started at http://127.26.250.189:37383/ using document root <none> and password file <none>
I20260812 03:10:48.624191 9369 fs_manager.cc:359] Metadata directory not provided
I20260812 03:10:48.624249 9369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:10:48.624421 9369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:10:48.626623 9369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-1/data/instance:
uuid: "22144cdc93ca441fbb45f8a23b33b030"
format_stamp: "Formatted at 2026-08-12 03:10:48 on dist-test-slave-zpfg"
server_key: "e445b7de20dcf863ce5c5d742b366fde"
server_key_iv: "4fedff463a2a8f1d1465eb971955fe54"
server_key_version: "encryptionkey@0"
I20260812 03:10:48.627166 9369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-1/wal/instance:
uuid: "22144cdc93ca441fbb45f8a23b33b030"
format_stamp: "Formatted at 2026-08-12 03:10:48 on dist-test-slave-zpfg"
server_key: "e445b7de20dcf863ce5c5d742b366fde"
server_key_iv: "4fedff463a2a8f1d1465eb971955fe54"
server_key_version: "encryptionkey@0"
I20260812 03:10:48.630998 9369 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 03:10:48.633757 9384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:48.635227 9369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:10:48.635352 9369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-1/wal
uuid: "22144cdc93ca441fbb45f8a23b33b030"
format_stamp: "Formatted at 2026-08-12 03:10:48 on dist-test-slave-zpfg"
server_key: "e445b7de20dcf863ce5c5d742b366fde"
server_key_iv: "4fedff463a2a8f1d1465eb971955fe54"
server_key_version: "encryptionkey@0"
I20260812 03:10:48.635452 9369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:10:48.655448 9369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:10:48.658823 9369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:10:48.658998 9369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:10:48.666677 9369 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.189:33581
I20260812 03:10:48.667004 9436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.189:33581 every 8 connection(s)
I20260812 03:10:48.667743 9369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-1/data/info.pb
I20260812 03:10:48.668644 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 9369
I20260812 03:10:48.668802 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-1/wal/instance
I20260812 03:10:48.669144 27626 external_mini_cluster.cc:1428] Setting key ce6f9df40af6d249e476775e011c45f4
I20260812 03:10:48.671972 9437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:10:48.674633 9362 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.189" port: 33581 } attempt: 3
I20260812 03:10:48.677143 9437 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42369 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.250.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.188@KRBTEST.COM" created.
Aug 12 03:10:48 dist-test-slave-zpfg krb5kdc[9279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504248, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for kudu/127.26.250.189@KRBTEST.COM
Aug 12 03:10:48 dist-test-slave-zpfg krb5kdc[9279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504248, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.189@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:10:48.690893 9365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.675274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:33581 (local address 127.0.0.1:59158)
0812 03:10:48.675557 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.675576 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:48.675691 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:48.678985 (+ 3294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:48.678988 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:48.679150 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:10:48.679280 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:48.679286 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.681377 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.681382 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:48.682138 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:48.682148 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.683335 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.683340 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:48.683466 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:48.684071 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:10:48.684092 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:10:48.685862 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:10:48.688292 (+ 2430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:48.688299 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:48.688304 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:48.688734 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:48.689236 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:48.689241 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:48.689244 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:48.689350 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:48.690361 (+ 1011us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:10:48.690365 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:10:48.690492 (+ 127us) client_negotiation.cc:769] Sending connection context
0812 03:10:48.690603 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 03:10:48.690752 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168}
I20260812 03:10:48.692718 9366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.680004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50752 (local address 127.26.250.190:42369)
0812 03:10:48.680141 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:48.680146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:48.680564 (+ 418us) server_negotiation.cc:407] Connection header received
0812 03:10:48.680807 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:48.680811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:48.680874 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:48.680957 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:48.681980 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.682818 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.684185 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.684621 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.684741 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:48.687840 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:48.687848 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:48.687854 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:48.687900 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:48.689696 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:48.690181 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:48.690186 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:48.690188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:48.690267 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:48.690647 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:48.690654 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:48.690658 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:48.690959 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:48.691099 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:48.692450 (+ 1351us) server_negotiation.cc:299] Negotiation successful
0812 03:10:48.692581 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 03:10:48.693087 9441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.679910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:42369 (local address 127.0.0.1:50752)
0812 03:10:48.680401 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.680456 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:48.680655 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:48.681130 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:48.681140 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:48.681649 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:10:48.681841 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:48.681850 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.682979 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.682983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:48.684044 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:48.684054 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.684944 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.684951 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:48.685101 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:48.685665 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:10:48.685683 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:10:48.687652 (+ 1969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:10:48.689801 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:48.689812 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:48.689819 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:48.690075 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:48.690431 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:48.690434 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:48.690436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:48.690527 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:48.691269 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:10:48.691275 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:10:48.691389 (+ 114us) client_negotiation.cc:769] Sending connection context
0812 03:10:48.691537 (+ 148us) client_negotiation.cc:240] Negotiation successful
0812 03:10:48.691895 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":68,"thread_start_us":171,"threads_started":1}
I20260812 03:10:48.693090 9440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.676589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59158 (local address 127.26.250.189:33581)
0812 03:10:48.677143 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:48.677166 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:48.677216 (+ 50us) server_negotiation.cc:407] Connection header received
0812 03:10:48.678063 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:48.678092 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:48.678596 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:48.679018 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:48.679506 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.681234 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.682307 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.683180 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.683803 (+ 623us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:48.686038 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:48.686046 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:48.686060 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:48.686085 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:48.688143 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:48.688902 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:48.688914 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:48.688925 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:48.689099 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:48.689467 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:48.689474 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:48.689479 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:48.690196 (+ 717us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:48.690396 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:48.691357 (+ 961us) server_negotiation.cc:299] Negotiation successful
0812 03:10:48.691749 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":161,"threads_started":1}
I20260812 03:10:48.695396 9437 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.189" port: 33581 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:10:48.695647 9362 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:10:48.696511 9365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.696224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:39301 (local address 127.0.0.1:35494)
0812 03:10:48.696379 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.696444 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:39301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 03:10:48.697461 9365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.697139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:39301 (local address 127.0.0.1:35504)
0812 03:10:48.697322 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.697392 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:39301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260812 03:10:48.697628 9362 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.188:39301: Network error: Client connection negotiation failed: client connection to 127.26.250.188:39301: connect: Connection refused (error 111)
Aug 12 03:10:48 dist-test-slave-zpfg krb5kdc[9279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504248, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.189@KRBTEST.COM for kudu/127.26.250.189@KRBTEST.COM
I20260812 03:10:48.705554 9441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.695982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:33581 (local address 127.0.0.1:59160)
0812 03:10:48.696172 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.696186 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:48.696283 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:48.696937 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:48.696940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:48.697125 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:10:48.697250 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:48.697258 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.698248 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.698251 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:48.698964 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:48.698975 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.699500 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.699503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:48.699632 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:48.700197 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:10:48.700211 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:10:48.701697 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:10:48.703898 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:48.703903 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:48.703906 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:48.704110 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:48.704419 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:48.704425 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:48.704427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:48.704512 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:48.704970 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:10:48.704973 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:10:48.705071 (+ 98us) client_negotiation.cc:769] Sending connection context
0812 03:10:48.705241 (+ 170us) client_negotiation.cc:240] Negotiation successful
0812 03:10:48.705381 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":3}
I20260812 03:10:48.705637 9440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.696231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59160 (local address 127.26.250.189:33581)
0812 03:10:48.696712 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:48.696717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:48.696733 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:10:48.696777 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:48.696779 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:48.696815 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:48.696914 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:48.697382 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.698113 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.699077 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.699375 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.699488 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:48.701841 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:48.701844 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:48.701847 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:48.701875 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:48.703750 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:48.704217 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:48.704222 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:48.704224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:48.704325 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:48.704608 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:48.704610 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:48.704612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:48.704860 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:48.704981 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:48.705340 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 03:10:48.705465 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":411}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.188.keytab.
Entry for principal kudu/127.26.250.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.188.keytab.
I20260812 03:10:48.706637 9437 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:10:48.707114 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.188:39301
--webserver_interface=127.26.250.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.188.keytab
--principal=kudu/127.26.250.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:44175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:42369,127.26.250.189:33581,127.26.250.188:39301
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:10:48.707644 9441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.707351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:39301 (local address 127.0.0.1:35512)
0812 03:10:48.707492 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.707581 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:39301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 03:10:48.708300 9385 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.250.188:39301: connect: Connection refused (error 111)
I20260812 03:10:48.709444 9441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.709198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:39301 (local address 127.0.0.1:35518)
0812 03:10:48.709323 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.709374 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:39301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 03:10:48.709735 9437 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.188:39301: Network error: Client connection negotiation failed: client connection to 127.26.250.188:39301: connect: Connection refused (error 111)
I20260812 03:10:48.728827 9362 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } attempt: 1
I20260812 03:10:48.729944 9365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.729603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:39301 (local address 127.0.0.1:35524)
0812 03:10:48.729789 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.729859 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:39301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260812 03:10:48.731024 9365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.730661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:39301 (local address 127.0.0.1:35530)
0812 03:10:48.730858 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.730933 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:39301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260812 03:10:48.731182 9362 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.188:39301: Network error: Client connection negotiation failed: client connection to 127.26.250.188:39301: connect: Connection refused (error 111)
I20260812 03:10:48.759070 9437 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } attempt: 1
I20260812 03:10:48.760144 9441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.759815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:39301 (local address 127.0.0.1:35538)
0812 03:10:48.759990 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.760060 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:39301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260812 03:10:48.761027 9441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.760715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:39301 (local address 127.0.0.1:35550)
0812 03:10:48.760905 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.760964 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:39301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260812 03:10:48.761170 9437 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.188:39301: Network error: Client connection negotiation failed: client connection to 127.26.250.188:39301: connect: Connection refused (error 111)
I20260812 03:10:48.806519 9362 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } attempt: 2
I20260812 03:10:48.807490 9365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.807078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:39301 (local address 127.0.0.1:35558)
0812 03:10:48.807250 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.807403 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:39301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260812 03:10:48.808326 9365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.808066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:39301 (local address 127.0.0.1:35564)
0812 03:10:48.808190 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.808256 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:39301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 03:10:48.808468 9362 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.188:39301: Network error: Client connection negotiation failed: client connection to 127.26.250.188:39301: connect: Connection refused (error 111)
I20260812 03:10:48.829362 9437 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } attempt: 2
I20260812 03:10:48.830304 9441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.829968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:39301 (local address 127.0.0.1:35568)
0812 03:10:48.830136 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.830218 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:39301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 03:10:48.831354 9441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.831009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:39301 (local address 127.0.0.1:35572)
0812 03:10:48.831201 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.831269 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:39301: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
W20260812 03:10:48.831579 9437 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.188:39301: Network error: Client connection negotiation failed: client connection to 127.26.250.188:39301: connect: Connection refused (error 111)
W20260812 03:10:48.836407 9444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:10:48.836726 9444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:10:48.836769 9444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:10:48.841176 9444 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:10:48.841266 9444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:10:48.841289 9444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:10:48.841308 9444 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:10:48.841324 9444 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:10:48.847352 9444 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:44175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:42369,127.26.250.189:33581,127.26.250.188:39301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.188.keytab
--principal=kudu/127.26.250.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.188:39301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.250.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:10:48.848608 9444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:10:48.849749 9444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:10:48.860144 9452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:48.860126 9450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:48.860517 9444 server_base.cc:1034] running on GCE node
W20260812 03:10:48.860585 9449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:48.861232 9444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:10:48.862648 9444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:10:48.863833 9444 hybrid_clock.cc:648] HybridClock initialized: now 1786504248863815 us; error 33 us; skew 500 ppm
Aug 12 03:10:48 dist-test-slave-zpfg krb5kdc[9279](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504248, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:10:48.867101 9444 init.cc:377] Logged in from keytab as kudu/127.26.250.188@KRBTEST.COM (short username kudu)
I20260812 03:10:48.868237 9444 webserver.cc:460] Webserver started at http://127.26.250.188:39349/ using document root <none> and password file <none>
I20260812 03:10:48.868877 9444 fs_manager.cc:359] Metadata directory not provided
I20260812 03:10:48.868976 9444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:10:48.869228 9444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:10:48.871551 9444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-2/data/instance:
uuid: "f2677b5e05bf415a86d62d763cff0f1f"
format_stamp: "Formatted at 2026-08-12 03:10:48 on dist-test-slave-zpfg"
server_key: "ca8a9a8511e4cbdca4135af7d71cd548"
server_key_iv: "b53577e7db9c264a1b7b4f9812a34faa"
server_key_version: "encryptionkey@0"
I20260812 03:10:48.872174 9444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-2/wal/instance:
uuid: "f2677b5e05bf415a86d62d763cff0f1f"
format_stamp: "Formatted at 2026-08-12 03:10:48 on dist-test-slave-zpfg"
server_key: "ca8a9a8511e4cbdca4135af7d71cd548"
server_key_iv: "b53577e7db9c264a1b7b4f9812a34faa"
server_key_version: "encryptionkey@0"
I20260812 03:10:48.876514 9444 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:10:48.879287 9459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:48.880738 9444 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:10:48.880913 9444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-2/wal
uuid: "f2677b5e05bf415a86d62d763cff0f1f"
format_stamp: "Formatted at 2026-08-12 03:10:48 on dist-test-slave-zpfg"
server_key: "ca8a9a8511e4cbdca4135af7d71cd548"
server_key_iv: "b53577e7db9c264a1b7b4f9812a34faa"
server_key_version: "encryptionkey@0"
I20260812 03:10:48.881054 9444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:10:48.893819 9444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:10:48.897713 9444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:10:48.897987 9444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:10:48.905998 9444 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.188:39301
I20260812 03:10:48.906469 9514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.188:39301 every 8 connection(s)
I20260812 03:10:48.907207 9444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-2/data/info.pb
I20260812 03:10:48.907737 9362 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } attempt: 3
I20260812 03:10:48.911087 9515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:10:48.916082 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 9444
I20260812 03:10:48.916263 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786504075277362-27626-0/minicluster-data/master-2/wal/instance
I20260812 03:10:48.916638 27626 external_mini_cluster.cc:1428] Setting key e0a0b0af3bcee1f68e3970ddfd36ff62
I20260812 03:10:48.917141 9515 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42369 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:10:48 dist-test-slave-zpfg krb5kdc[9279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504248, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for kudu/127.26.250.188@KRBTEST.COM
I20260812 03:10:48.922792 9437 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } attempt: 3
I20260812 03:10:48.923974 9365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.908268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:39301 (local address 127.0.0.1:35576)
0812 03:10:48.908423 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.908440 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:48.908534 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:48.911623 (+ 3089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:48.911626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:48.911801 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:10:48.911945 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:48.911951 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.914078 (+ 2127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.914082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:48.914702 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:48.914721 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.915852 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.915859 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:48.915973 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:48.916580 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:10:48.916600 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:10:48.918236 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:10:48.921687 (+ 3451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:48.921691 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:48.921693 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:48.922029 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:48.922495 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:48.922501 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:48.922503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:48.922618 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:48.923422 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:10:48.923425 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:10:48.923524 (+ 99us) client_negotiation.cc:769] Sending connection context
0812 03:10:48.923612 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 03:10:48.923765 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
Aug 12 03:10:48 dist-test-slave-zpfg krb5kdc[9279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.188@KRBTEST.COM
I20260812 03:10:48.925976 9517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.909154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35576 (local address 127.26.250.188:39301)
0812 03:10:48.909607 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:48.909629 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:48.909677 (+ 48us) server_negotiation.cc:407] Connection header received
0812 03:10:48.910708 (+ 1031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:48.910736 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:48.911244 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:48.911676 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:48.912150 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.913913 (+ 1763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.914839 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.915660 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.915874 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:48.918500 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:48.918507 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:48.918525 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:48.918560 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:48.921545 (+ 2985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:48.922166 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:48.922176 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:48.922185 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:48.922390 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:48.922749 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:48.922752 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:48.922754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:48.923291 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:48.924296 (+ 1005us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:48.924490 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 03:10:48.924825 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":299,"thread_start_us":112,"threads_started":1}
Aug 12 03:10:48 dist-test-slave-zpfg krb5kdc[9279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504248, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.188@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:10:48.931116 9520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.919557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:42369 (local address 127.0.0.1:50754)
0812 03:10:48.919977 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.920036 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:48.920216 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:48.920548 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:48.920553 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:48.921112 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:10:48.921282 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:48.921291 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.922210 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.922212 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:48.923250 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:48.923260 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.924300 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.924304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:48.924427 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:48.925008 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:10:48.925020 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:10:48.926817 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:10:48.928942 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:48.928952 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:48.928959 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:48.929284 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:48.929620 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:48.929623 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:48.929625 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:48.929725 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:48.930296 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:10:48.930302 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:10:48.930470 (+ 168us) client_negotiation.cc:769] Sending connection context
0812 03:10:48.930624 (+ 154us) client_negotiation.cc:240] Negotiation successful
0812 03:10:48.930861 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":29,"thread_start_us":158,"threads_started":1}
Aug 12 03:10:48 dist-test-slave-zpfg krb5kdc[9279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504248, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.189@KRBTEST.COM for kudu/127.26.250.188@KRBTEST.COM
I20260812 03:10:48.932106 9366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.919593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50754 (local address 127.26.250.190:42369)
0812 03:10:48.919747 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:48.919752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:48.920118 (+ 366us) server_negotiation.cc:407] Connection header received
0812 03:10:48.920349 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:48.920354 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:48.920414 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:48.920500 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:48.921444 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.922059 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.923397 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.924062 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.924145 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:48.926996 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:48.927000 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:48.927002 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:48.927034 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:48.928811 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:48.929423 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:48.929427 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:48.929429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:48.929518 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:48.929826 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:48.929829 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:48.929844 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:48.930127 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:48.930240 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:48.931791 (+ 1551us) server_negotiation.cc:299] Negotiation successful
0812 03:10:48.931947 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 03:10:48.931938 9519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.917910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35590 (local address 127.26.250.188:39301)
0812 03:10:48.918325 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:48.918334 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:48.918390 (+ 56us) server_negotiation.cc:407] Connection header received
0812 03:10:48.918430 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:48.918433 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:48.918482 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:48.918589 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:48.919513 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.920700 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.922196 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.922654 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.922807 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:48.927034 (+ 4227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:48.927038 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:48.927041 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:48.927092 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:48.929121 (+ 2029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:48.929770 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:48.929773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:48.929775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:48.929893 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:48.930267 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:48.930270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:48.930272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:48.930571 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:48.931462 (+ 891us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:48.931550 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 03:10:48.931718 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":324,"thread_start_us":195,"threads_started":1}
I20260812 03:10:48.933614 9515 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.189" port: 33581 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:10:48.934485 9362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72361de1616e4b56b5e96a3d7933bb22: Bootstrap starting.
I20260812 03:10:48.936908 9441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.923385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:39301 (local address 127.0.0.1:35600)
0812 03:10:48.923735 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.923749 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:48.923858 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:48.925795 (+ 1937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:48.925799 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:48.926010 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:10:48.926130 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:48.926138 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.928722 (+ 2584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.928731 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:48.929597 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:48.929612 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.930228 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.930232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:48.930322 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:48.930906 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:10:48.930921 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:10:48.932490 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:10:48.935306 (+ 2816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:48.935310 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:48.935312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:48.935545 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:48.935833 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:48.935836 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:48.935838 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:48.935927 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:48.936354 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:10:48.936357 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:10:48.936465 (+ 108us) client_negotiation.cc:769] Sending connection context
0812 03:10:48.936573 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 03:10:48.936739 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241}
I20260812 03:10:48.936926 9521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.924026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35600 (local address 127.26.250.188:39301)
0812 03:10:48.924811 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:48.924865 (+ 54us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:48.924878 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:10:48.924911 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:48.924914 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:48.925636 (+ 722us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:48.926255 (+ 619us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:48.926381 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.928549 (+ 2168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.929743 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.930076 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.930602 (+ 526us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:48.932646 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:48.932653 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:48.932657 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:48.932684 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:48.935180 (+ 2496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:48.935650 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:48.935653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:48.935655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:48.935723 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:48.936032 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:48.936035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:48.936037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:48.936246 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:48.936361 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:48.936637 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 03:10:48.936754 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":145,"threads_started":1}
I20260812 03:10:48.940420 9437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030: Bootstrap starting.
Aug 12 03:10:48 dist-test-slave-zpfg krb5kdc[9279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:10:48.942047 9362 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72361de1616e4b56b5e96a3d7933bb22: Neither blocks nor log segments found. Creating new log.
I20260812 03:10:48.943475 9437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030: Neither blocks nor log segments found. Creating new log.
I20260812 03:10:48.943650 9362 log.cc:824] T 00000000000000000000000000000000 P 72361de1616e4b56b5e96a3d7933bb22: Log is configured to *not* fsync() on all Append() calls
Aug 12 03:10:48 dist-test-slave-zpfg krb5kdc[9279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504248, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.188@KRBTEST.COM for kudu/127.26.250.189@KRBTEST.COM
I20260812 03:10:48.944397 9437 log.cc:824] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030: Log is configured to *not* fsync() on all Append() calls
I20260812 03:10:48.946501 9362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72361de1616e4b56b5e96a3d7933bb22: No bootstrap required, opened a new log
I20260812 03:10:48.946820 9366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.934839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50762 (local address 127.26.250.190:42369)
0812 03:10:48.934967 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:48.934973 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:48.934989 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:10:48.935064 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:48.935068 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:48.935112 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:48.935215 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:48.936112 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.936965 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.938125 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.938492 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.938617 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:48.941843 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:48.941847 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:48.941850 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:48.941881 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:48.944486 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:48.945058 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:48.945061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:48.945063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:48.945148 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:48.945442 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:48.945445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:48.945446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:48.945679 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:48.945852 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:48.946537 (+ 685us) server_negotiation.cc:299] Negotiation successful
0812 03:10:48.946677 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:10:48.946870 9437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030: No bootstrap required, opened a new log
I20260812 03:10:48.949224 9440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.934506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59162 (local address 127.26.250.189:33581)
0812 03:10:48.934646 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:48.934651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:48.934667 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:10:48.934716 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:48.934719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:48.934775 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:48.934868 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:48.939687 (+ 4819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.940657 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.941961 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.942369 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.942499 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:48.945059 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:48.945063 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:48.945067 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:48.945099 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:48.947026 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:48.947765 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:48.947769 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:48.947771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:48.947858 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:48.948188 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:48.948192 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:48.948194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:48.948454 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:48.948530 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:48.948931 (+ 401us) server_negotiation.cc:299] Negotiation successful
0812 03:10:48.949058 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 03:10:48.949708 9520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.934209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:33581 (local address 127.0.0.1:59162)
0812 03:10:48.934518 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.934533 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:48.934618 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:48.938757 (+ 4139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:48.938761 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:48.939367 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:10:48.939516 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:48.939524 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.940829 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.940837 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:48.941813 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:48.941824 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.942493 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.942497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:48.942582 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:48.943145 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:10:48.943161 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:10:48.944878 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:10:48.947349 (+ 2471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:48.947353 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:48.947355 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:48.947635 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:48.947978 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:48.947984 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:48.947988 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:48.948078 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:48.948610 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:10:48.948613 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:10:48.948726 (+ 113us) client_negotiation.cc:769] Sending connection context
0812 03:10:48.949357 (+ 631us) client_negotiation.cc:240] Negotiation successful
0812 03:10:48.949552 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225}
I20260812 03:10:48.950572 9515 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:10:48.951673 9362 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72361de1616e4b56b5e96a3d7933bb22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72361de1616e4b56b5e96a3d7933bb22" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42369 } } peers { permanent_uuid: "22144cdc93ca441fbb45f8a23b33b030" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 33581 } } peers { permanent_uuid: "f2677b5e05bf415a86d62d763cff0f1f" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } }
I20260812 03:10:48.951890 9362 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72361de1616e4b56b5e96a3d7933bb22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:10:48.951964 9362 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72361de1616e4b56b5e96a3d7933bb22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72361de1616e4b56b5e96a3d7933bb22, State: Initialized, Role: FOLLOWER
I20260812 03:10:48.952484 9362 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72361de1616e4b56b5e96a3d7933bb22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72361de1616e4b56b5e96a3d7933bb22" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42369 } } peers { permanent_uuid: "22144cdc93ca441fbb45f8a23b33b030" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 33581 } } peers { permanent_uuid: "f2677b5e05bf415a86d62d763cff0f1f" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } }
I20260812 03:10:48.952564 9437 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72361de1616e4b56b5e96a3d7933bb22" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42369 } } peers { permanent_uuid: "22144cdc93ca441fbb45f8a23b33b030" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 33581 } } peers { permanent_uuid: "f2677b5e05bf415a86d62d763cff0f1f" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } }
I20260812 03:10:48.952910 9437 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:10:48.953037 9437 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22144cdc93ca441fbb45f8a23b33b030, State: Initialized, Role: FOLLOWER
I20260812 03:10:48.953615 9437 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72361de1616e4b56b5e96a3d7933bb22" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42369 } } peers { permanent_uuid: "22144cdc93ca441fbb45f8a23b33b030" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 33581 } } peers { permanent_uuid: "f2677b5e05bf415a86d62d763cff0f1f" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } }
I20260812 03:10:48.954025 9362 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72361de1616e4b56b5e96a3d7933bb22 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:10:48.955358 9437 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [sys.catalog]: configured and running, proceeding with master startup.
Aug 12 03:10:48 dist-test-slave-zpfg krb5kdc[9279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504248, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.188@KRBTEST.COM for kudu/127.26.250.188@KRBTEST.COM
Aug 12 03:10:48 dist-test-slave-zpfg krb5kdc[9279](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.189@KRBTEST.COM
W20260812 03:10:48.961331 9539 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 72361de1616e4b56b5e96a3d7933bb22: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:10:48.961414 9539 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:10:48.961686 9520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.951079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:39301 (local address 127.0.0.1:35616)
0812 03:10:48.951221 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:48.951236 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:48.951311 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:48.952386 (+ 1075us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:48.952389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:48.952579 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:10:48.952686 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:48.952694 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.953691 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.953694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:48.954455 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:48.954468 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.955183 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.955190 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:48.955280 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:48.955774 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:10:48.955790 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:10:48.958078 (+ 2288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:10:48.960131 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:48.960134 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:48.960136 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:48.960369 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:48.960633 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:48.960635 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:48.960637 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:48.960726 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:48.961142 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:10:48.961145 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:10:48.961225 (+ 80us) client_negotiation.cc:769] Sending connection context
0812 03:10:48.961321 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 03:10:48.961476 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 03:10:48.961669 9525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72361de1616e4b56b5e96a3d7933bb22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72361de1616e4b56b5e96a3d7933bb22" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42369 } } peers { permanent_uuid: "22144cdc93ca441fbb45f8a23b33b030" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 33581 } } peers { permanent_uuid: "f2677b5e05bf415a86d62d763cff0f1f" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } } }
I20260812 03:10:48.961784 9525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72361de1616e4b56b5e96a3d7933bb22 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:10:48.962126 9521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.951277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35616 (local address 127.26.250.188:39301)
0812 03:10:48.952074 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:48.952079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:48.952120 (+ 41us) server_negotiation.cc:407] Connection header received
0812 03:10:48.952156 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:48.952160 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:48.952203 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:48.952283 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:48.952827 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.953554 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.954587 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.954942 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.955057 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:48.958222 (+ 3165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:48.958225 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:48.958228 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:48.958249 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:48.960013 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:48.960470 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:48.960473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:48.960474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:48.960546 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:48.960823 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:48.960826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:48.960828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:48.961024 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:48.961110 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:48.961880 (+ 770us) server_negotiation.cc:299] Negotiation successful
0812 03:10:48.961992 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":718}
I20260812 03:10:48.962836 9526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72361de1616e4b56b5e96a3d7933bb22" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42369 } } peers { permanent_uuid: "22144cdc93ca441fbb45f8a23b33b030" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 33581 } } peers { permanent_uuid: "f2677b5e05bf415a86d62d763cff0f1f" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } } }
I20260812 03:10:48.963374 9526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:10:48.965584 9515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2677b5e05bf415a86d62d763cff0f1f: Bootstrap starting.
I20260812 03:10:48.968554 9515 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2677b5e05bf415a86d62d763cff0f1f: Neither blocks nor log segments found. Creating new log.
I20260812 03:10:48.964761 9369 master_runner.cc:186] Error getting master registration for 127.26.250.188:39301: OK, instance_id { permanent_uuid: "f2677b5e05bf415a86d62d763cff0f1f" instance_seqno: 1786504248903776 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:10:48.969406 9369 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:10:48.969580 9515 log.cc:824] T 00000000000000000000000000000000 P f2677b5e05bf415a86d62d763cff0f1f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:10:48.972204 9515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2677b5e05bf415a86d62d763cff0f1f: No bootstrap required, opened a new log
I20260812 03:10:48.968768 9292 master_runner.cc:186] Error getting master registration for 127.26.250.188:39301: OK, instance_id { permanent_uuid: "f2677b5e05bf415a86d62d763cff0f1f" instance_seqno: 1786504248903776 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:10:48.973255 9292 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:10:48.975730 9524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.948579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59170 (local address 127.26.250.189:33581)
0812 03:10:48.949217 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:48.949226 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:48.949243 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:10:48.949283 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:48.949286 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:48.949335 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:48.950262 (+ 927us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:48.950464 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.951887 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.954011 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.954441 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.957009 (+ 2568us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:48.958794 (+ 1785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:48.958804 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:48.958808 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:48.958839 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:48.972634 (+ 13795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:48.973378 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:48.973385 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:48.973390 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:48.973505 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:48.973896 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:48.973900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:48.973901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:48.974157 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:48.975252 (+ 1095us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:48.975363 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 03:10:48.975526 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":110,"threads_started":1}
W20260812 03:10:48.977449 9550 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:10:48.977573 9550 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:10:48.977560 9515 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2677b5e05bf415a86d62d763cff0f1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72361de1616e4b56b5e96a3d7933bb22" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42369 } } peers { permanent_uuid: "22144cdc93ca441fbb45f8a23b33b030" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 33581 } } peers { permanent_uuid: "f2677b5e05bf415a86d62d763cff0f1f" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } }
I20260812 03:10:48.977892 9515 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2677b5e05bf415a86d62d763cff0f1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:10:48.977972 9515 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2677b5e05bf415a86d62d763cff0f1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2677b5e05bf415a86d62d763cff0f1f, State: Initialized, Role: FOLLOWER
I20260812 03:10:48.978546 9515 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2677b5e05bf415a86d62d763cff0f1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72361de1616e4b56b5e96a3d7933bb22" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42369 } } peers { permanent_uuid: "22144cdc93ca441fbb45f8a23b33b030" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 33581 } } peers { permanent_uuid: "f2677b5e05bf415a86d62d763cff0f1f" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } }
I20260812 03:10:48.979830 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:10:48.980226 9515 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2677b5e05bf415a86d62d763cff0f1f [sys.catalog]: configured and running, proceeding with master startup.
W20260812 03:10:48.985670 9568 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f2677b5e05bf415a86d62d763cff0f1f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:10:48.985877 9568 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:10:48.989131 9444 master_runner.cc:263] No leader master found from master f2677b5e05bf415a86d62d763cff0f1f
I20260812 03:10:48.989228 9444 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:10:48.997033 9526 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:10:48.997323 9526 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72361de1616e4b56b5e96a3d7933bb22" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42369 } } peers { permanent_uuid: "22144cdc93ca441fbb45f8a23b33b030" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 33581 } } peers { permanent_uuid: "f2677b5e05bf415a86d62d763cff0f1f" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } }
I20260812 03:10:48.998924 9526 leader_election.cc:288] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72361de1616e4b56b5e96a3d7933bb22 (127.26.250.190:42369), f2677b5e05bf415a86d62d763cff0f1f (127.26.250.188:39301)
I20260812 03:10:49.000423 9336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "22144cdc93ca441fbb45f8a23b33b030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72361de1616e4b56b5e96a3d7933bb22" is_pre_election: true
I20260812 03:10:49.000969 9336 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 72361de1616e4b56b5e96a3d7933bb22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22144cdc93ca441fbb45f8a23b33b030 in term 0.
I20260812 03:10:49.001646 9488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "22144cdc93ca441fbb45f8a23b33b030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2677b5e05bf415a86d62d763cff0f1f" is_pre_election: true
I20260812 03:10:49.002084 9488 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f2677b5e05bf415a86d62d763cff0f1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22144cdc93ca441fbb45f8a23b33b030 in term 0.
I20260812 03:10:49.003182 9385 leader_election.cc:302] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22144cdc93ca441fbb45f8a23b33b030, f2677b5e05bf415a86d62d763cff0f1f; no voters:
I20260812 03:10:49.004124 9526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:10:49.004271 9526 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:10:49.004336 9526 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:10:49.004756 9521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.984674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35620 (local address 127.26.250.188:39301)
0812 03:10:48.984791 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:48.984796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:48.993408 (+ 8612us) server_negotiation.cc:407] Connection header received
0812 03:10:48.993598 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:48.993602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:48.993655 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:48.993725 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:48.994913 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.995834 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.997164 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.997508 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.997597 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:48.999885 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:48.999892 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:48.999895 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:48.999931 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:49.002067 (+ 2136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:49.002853 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:49.002858 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:49.002860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:49.002975 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:49.003378 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:49.003382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:49.003384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:49.003703 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:49.003889 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:49.004349 (+ 460us) server_negotiation.cc:299] Negotiation successful
0812 03:10:49.004513 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:10:49.005160 9553 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2677b5e05bf415a86d62d763cff0f1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72361de1616e4b56b5e96a3d7933bb22" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42369 } } peers { permanent_uuid: "22144cdc93ca441fbb45f8a23b33b030" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 33581 } } peers { permanent_uuid: "f2677b5e05bf415a86d62d763cff0f1f" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } } }
I20260812 03:10:49.005401 9553 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2677b5e05bf415a86d62d763cff0f1f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:10:49.006757 9526 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72361de1616e4b56b5e96a3d7933bb22" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42369 } } peers { permanent_uuid: "22144cdc93ca441fbb45f8a23b33b030" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 33581 } } peers { permanent_uuid: "f2677b5e05bf415a86d62d763cff0f1f" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } }
I20260812 03:10:49.007786 9336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "22144cdc93ca441fbb45f8a23b33b030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72361de1616e4b56b5e96a3d7933bb22"
I20260812 03:10:49.008008 9336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72361de1616e4b56b5e96a3d7933bb22 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:10:49.009327 9336 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 72361de1616e4b56b5e96a3d7933bb22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22144cdc93ca441fbb45f8a23b33b030 in term 1.
I20260812 03:10:49.009701 9526 leader_election.cc:288] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [CANDIDATE]: Term 1 election: Requested vote from peers 72361de1616e4b56b5e96a3d7933bb22 (127.26.250.190:42369), f2677b5e05bf415a86d62d763cff0f1f (127.26.250.188:39301)
I20260812 03:10:49.009764 9385 leader_election.cc:302] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22144cdc93ca441fbb45f8a23b33b030, 72361de1616e4b56b5e96a3d7933bb22; no voters:
I20260812 03:10:49.009958 9526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:10:49.010409 9488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "22144cdc93ca441fbb45f8a23b33b030" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2677b5e05bf415a86d62d763cff0f1f"
I20260812 03:10:49.010517 9526 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [term 1 LEADER]: Becoming Leader. State: Replica: 22144cdc93ca441fbb45f8a23b33b030, State: Running, Role: LEADER
I20260812 03:10:49.010583 9488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2677b5e05bf415a86d62d763cff0f1f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:10:49.010972 9526 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72361de1616e4b56b5e96a3d7933bb22" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42369 } } peers { permanent_uuid: "22144cdc93ca441fbb45f8a23b33b030" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 33581 } } peers { permanent_uuid: "f2677b5e05bf415a86d62d763cff0f1f" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } }
I20260812 03:10:49.012248 9488 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f2677b5e05bf415a86d62d763cff0f1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22144cdc93ca441fbb45f8a23b33b030 in term 1.
I20260812 03:10:49.013726 9526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22144cdc93ca441fbb45f8a23b33b030. Latest consensus state: current_term: 1 leader_uuid: "22144cdc93ca441fbb45f8a23b33b030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72361de1616e4b56b5e96a3d7933bb22" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42369 } } peers { permanent_uuid: "22144cdc93ca441fbb45f8a23b33b030" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 33581 } } peers { permanent_uuid: "f2677b5e05bf415a86d62d763cff0f1f" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } } }
I20260812 03:10:49.013943 9526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [sys.catalog]: This master's current role is: LEADER
I20260812 03:10:49.022763 9576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:10:49.024416 9576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:10:49.027525 9366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.983933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50776 (local address 127.26.250.190:42369)
0812 03:10:48.984050 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:48.984055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:48.990585 (+ 6530us) server_negotiation.cc:407] Connection header received
0812 03:10:48.990777 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:48.990782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:48.990839 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:48.990900 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:48.992001 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.992837 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:49.014314 (+ 21477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:49.014764 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:49.014877 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:49.018696 (+ 3819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:49.018702 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:49.018706 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:49.018747 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:49.020885 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:49.024441 (+ 3556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:49.024448 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:49.024450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:49.024574 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:49.024982 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:49.024987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:49.024989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:49.025271 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:49.025407 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:49.027160 (+ 1753us) server_negotiation.cc:299] Negotiation successful
0812 03:10:49.027346 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:10:49.028895 9524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:48.982924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59184 (local address 127.26.250.189:33581)
0812 03:10:48.983512 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:48.983518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:48.985447 (+ 1929us) server_negotiation.cc:407] Connection header received
0812 03:10:48.985686 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:48.985690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:48.985747 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:48.985867 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:48.990180 (+ 4313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.991087 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.992410 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:48.992822 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:48.992921 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:49.021594 (+ 28673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:49.021599 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:49.021603 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:49.021643 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:49.023770 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:49.025800 (+ 2030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:49.025805 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:49.025807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:49.025914 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:49.026271 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:49.026276 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:49.026278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:49.026553 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:49.026722 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:49.028502 (+ 1780us) server_negotiation.cc:299] Negotiation successful
0812 03:10:49.028661 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493}
I20260812 03:10:49.041776 9488 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f2677b5e05bf415a86d62d763cff0f1f [term 1 FOLLOWER]: Refusing update from remote peer 22144cdc93ca441fbb45f8a23b33b030: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:10:49.042801 9336 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 72361de1616e4b56b5e96a3d7933bb22 [term 1 FOLLOWER]: Refusing update from remote peer 22144cdc93ca441fbb45f8a23b33b030: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:10:49.043812 9526 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2677b5e05bf415a86d62d763cff0f1f" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:10:49.044335 9526 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72361de1616e4b56b5e96a3d7933bb22" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:10:49.048864 9553 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2677b5e05bf415a86d62d763cff0f1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22144cdc93ca441fbb45f8a23b33b030. Latest consensus state: current_term: 1 leader_uuid: "22144cdc93ca441fbb45f8a23b33b030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72361de1616e4b56b5e96a3d7933bb22" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42369 } } peers { permanent_uuid: "22144cdc93ca441fbb45f8a23b33b030" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 33581 } } peers { permanent_uuid: "f2677b5e05bf415a86d62d763cff0f1f" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } } }
I20260812 03:10:49.049064 9553 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2677b5e05bf415a86d62d763cff0f1f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:10:49.050604 9525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72361de1616e4b56b5e96a3d7933bb22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22144cdc93ca441fbb45f8a23b33b030. Latest consensus state: current_term: 1 leader_uuid: "22144cdc93ca441fbb45f8a23b33b030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72361de1616e4b56b5e96a3d7933bb22" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42369 } } peers { permanent_uuid: "22144cdc93ca441fbb45f8a23b33b030" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 33581 } } peers { permanent_uuid: "f2677b5e05bf415a86d62d763cff0f1f" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } } }
I20260812 03:10:49.050751 9525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72361de1616e4b56b5e96a3d7933bb22 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:10:49.064774 9574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "22144cdc93ca441fbb45f8a23b33b030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72361de1616e4b56b5e96a3d7933bb22" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42369 } } peers { permanent_uuid: "22144cdc93ca441fbb45f8a23b33b030" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 33581 } } peers { permanent_uuid: "f2677b5e05bf415a86d62d763cff0f1f" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } } }
I20260812 03:10:49.064966 9574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [sys.catalog]: This master's current role is: LEADER
I20260812 03:10:49.065267 9577 mvcc.cc:205] Tried to move back new op lower bound from 7317521404063875072 to 7317521403955957760. Current Snapshot: MvccSnapshot[applied={T|T < 7317521404063875072}]
I20260812 03:10:49.066387 9553 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2677b5e05bf415a86d62d763cff0f1f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "22144cdc93ca441fbb45f8a23b33b030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72361de1616e4b56b5e96a3d7933bb22" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42369 } } peers { permanent_uuid: "22144cdc93ca441fbb45f8a23b33b030" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 33581 } } peers { permanent_uuid: "f2677b5e05bf415a86d62d763cff0f1f" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } } }
I20260812 03:10:49.066533 9553 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2677b5e05bf415a86d62d763cff0f1f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:10:49.069137 9526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "22144cdc93ca441fbb45f8a23b33b030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72361de1616e4b56b5e96a3d7933bb22" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42369 } } peers { permanent_uuid: "22144cdc93ca441fbb45f8a23b33b030" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 33581 } } peers { permanent_uuid: "f2677b5e05bf415a86d62d763cff0f1f" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } } }
I20260812 03:10:49.069290 9526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030 [sys.catalog]: This master's current role is: LEADER
I20260812 03:10:49.070874 9525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72361de1616e4b56b5e96a3d7933bb22 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "22144cdc93ca441fbb45f8a23b33b030" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72361de1616e4b56b5e96a3d7933bb22" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42369 } } peers { permanent_uuid: "22144cdc93ca441fbb45f8a23b33b030" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 33581 } } peers { permanent_uuid: "f2677b5e05bf415a86d62d763cff0f1f" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 39301 } } }
I20260812 03:10:49.071009 9525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72361de1616e4b56b5e96a3d7933bb22 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:10:49.071537 9580 mvcc.cc:205] Tried to move back new op lower bound from 7317521404063875072 to 7317521403955957760. Current Snapshot: MvccSnapshot[applied={T|T < 7317521404063875072}]
I20260812 03:10:49.072022 9578 mvcc.cc:205] Tried to move back new op lower bound from 7317521404063875072 to 7317521403955957760. Current Snapshot: MvccSnapshot[applied={T|T < 7317521404063875072}]
I20260812 03:10:49.074525 9576 catalog_manager.cc:1347] Generated new cluster ID: e1b9f8e2532d4362a528e7bb7b5c8099
I20260812 03:10:49.074599 9576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:10:49.107935 9576 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:10:49.109532 9576 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:10:49.131886 9576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22144cdc93ca441fbb45f8a23b33b030: Generated new TSK 0
I20260812 03:10:49.133008 9576 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:10:49.962869 9539 catalog_manager.cc:1259] Loaded cluster ID: e1b9f8e2532d4362a528e7bb7b5c8099
I20260812 03:10:49.962958 9539 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 72361de1616e4b56b5e96a3d7933bb22: loading cluster ID for follower catalog manager: success
I20260812 03:10:49.964623 9539 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 72361de1616e4b56b5e96a3d7933bb22: acquiring CA information for follower catalog manager: success
I20260812 03:10:49.965904 9539 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 72361de1616e4b56b5e96a3d7933bb22: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:10:49.987296 9568 catalog_manager.cc:1259] Loaded cluster ID: e1b9f8e2532d4362a528e7bb7b5c8099
I20260812 03:10:49.987452 9568 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f2677b5e05bf415a86d62d763cff0f1f: loading cluster ID for follower catalog manager: success
I20260812 03:10:49.989156 9568 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f2677b5e05bf415a86d62d763cff0f1f: acquiring CA information for follower catalog manager: success
I20260812 03:10:49.990401 9568 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f2677b5e05bf415a86d62d763cff0f1f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:10:50.205724 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 9292
I20260812 03:10:50.216041 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 9369
I20260812 03:10:50.225903 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 9444
2026-08-12T03:10:50Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3218 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:10:50 dist-test-slave-zpfg krb5kdc[9592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:10:50 dist-test-slave-zpfg krb5kdc[9592](info): set up 2 sockets
Aug 12 03:10:50 dist-test-slave-zpfg krb5kdc[9592](info): commencing operation
krb5kdc: starting...
W20260812 03:10:51.432157 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:10:51 dist-test-slave-zpfg krb5kdc[9592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:10:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:10:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:10:51.577677 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:33667
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:41453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:33667
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:10:51.706475 9605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:10:51.706827 9605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:10:51.706909 9605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:10:51.711246 9605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:10:51.711361 9605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:10:51.711400 9605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:10:51.711441 9605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:10:51.711498 9605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:10:51.717468 9605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:41453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:33667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:33667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:10:51.718772 9605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:10:51.719888 9605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:10:51.729358 9612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:51.729358 9611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:51.729457 9614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:51.729852 9605 server_base.cc:1034] running on GCE node
I20260812 03:10:51.730633 9605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:10:51.731842 9605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:10:51.733048 9605 hybrid_clock.cc:648] HybridClock initialized: now 1786504251733030 us; error 34 us; skew 500 ppm
Aug 12 03:10:51 dist-test-slave-zpfg krb5kdc[9592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504251, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:10:51.736487 9605 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:10:51.737530 9605 webserver.cc:460] Webserver started at http://127.26.250.190:36169/ using document root <none> and password file <none>
I20260812 03:10:51.738080 9605 fs_manager.cc:359] Metadata directory not provided
I20260812 03:10:51.738142 9605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:10:51.738330 9605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:10:51.740590 9605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "64e0ccf2e66f4668afe30137efb5838f"
format_stamp: "Formatted at 2026-08-12 03:10:51 on dist-test-slave-zpfg"
server_key: "c3c72f85a9385d2a6be8b8cba285feac"
server_key_iv: "83cea92a02668602fe3ae09c0ad17fc7"
server_key_version: "encryptionkey@0"
I20260812 03:10:51.741194 9605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "64e0ccf2e66f4668afe30137efb5838f"
format_stamp: "Formatted at 2026-08-12 03:10:51 on dist-test-slave-zpfg"
server_key: "c3c72f85a9385d2a6be8b8cba285feac"
server_key_iv: "83cea92a02668602fe3ae09c0ad17fc7"
server_key_version: "encryptionkey@0"
I20260812 03:10:51.745460 9605 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260812 03:10:51.748755 9621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:51.750458 9605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:10:51.750608 9605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "64e0ccf2e66f4668afe30137efb5838f"
format_stamp: "Formatted at 2026-08-12 03:10:51 on dist-test-slave-zpfg"
server_key: "c3c72f85a9385d2a6be8b8cba285feac"
server_key_iv: "83cea92a02668602fe3ae09c0ad17fc7"
server_key_version: "encryptionkey@0"
I20260812 03:10:51.750749 9605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:10:51.768942 9605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:10:51.772447 9605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:10:51.772681 9605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:10:51.780957 9605 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:33667
I20260812 03:10:51.781028 9676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:33667 every 8 connection(s)
I20260812 03:10:51.782107 9605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:10:51.785430 9677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:10:51.786698 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 9605
I20260812 03:10:51.786804 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:10:51.787119 27626 external_mini_cluster.cc:1428] Setting key e9ed05af8312770041c292e188afd486
I20260812 03:10:51.792752 9677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64e0ccf2e66f4668afe30137efb5838f: Bootstrap starting.
I20260812 03:10:51.795924 9677 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64e0ccf2e66f4668afe30137efb5838f: Neither blocks nor log segments found. Creating new log.
I20260812 03:10:51.797041 9677 log.cc:824] T 00000000000000000000000000000000 P 64e0ccf2e66f4668afe30137efb5838f: Log is configured to *not* fsync() on all Append() calls
Aug 12 03:10:51 dist-test-slave-zpfg krb5kdc[9592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:10:51.799616 9677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64e0ccf2e66f4668afe30137efb5838f: No bootstrap required, opened a new log
I20260812 03:10:51.803525 9680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:51.788854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49792 (local address 127.26.250.190:33667)
0812 03:10:51.789329 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:51.789347 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:51.789387 (+ 40us) server_negotiation.cc:407] Connection header received
0812 03:10:51.790052 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:51.790071 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:51.790516 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:51.790820 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:51.791840 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:51.793046 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:51.794280 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:51.794949 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:51.795092 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:51.798323 (+ 3231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:51.798330 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:51.798394 (+ 64us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:51.798426 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:51.800391 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:51.800911 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:51.800923 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:51.800929 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:51.801051 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:51.801372 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:51.801376 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:51.801378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:51.801818 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:51.802024 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:51.802396 (+ 372us) server_negotiation.cc:299] Negotiation successful
0812 03:10:51.802718 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":154,"threads_started":1}
I20260812 03:10:51.803483 9677 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64e0ccf2e66f4668afe30137efb5838f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e0ccf2e66f4668afe30137efb5838f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 33667 } }
I20260812 03:10:51.803813 9677 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64e0ccf2e66f4668afe30137efb5838f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:10:51.803910 9677 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64e0ccf2e66f4668afe30137efb5838f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64e0ccf2e66f4668afe30137efb5838f, State: Initialized, Role: FOLLOWER
I20260812 03:10:51.804353 9677 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64e0ccf2e66f4668afe30137efb5838f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e0ccf2e66f4668afe30137efb5838f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 33667 } }
I20260812 03:10:51.804513 9677 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64e0ccf2e66f4668afe30137efb5838f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:10:51.804602 9677 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64e0ccf2e66f4668afe30137efb5838f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:10:51.804708 9677 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64e0ccf2e66f4668afe30137efb5838f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:10:51.805804 9677 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64e0ccf2e66f4668afe30137efb5838f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e0ccf2e66f4668afe30137efb5838f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 33667 } }
I20260812 03:10:51.806291 9677 leader_election.cc:302] T 00000000000000000000000000000000 P 64e0ccf2e66f4668afe30137efb5838f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64e0ccf2e66f4668afe30137efb5838f; no voters:
I20260812 03:10:51.806708 9677 leader_election.cc:288] T 00000000000000000000000000000000 P 64e0ccf2e66f4668afe30137efb5838f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:10:51.806808 9682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64e0ccf2e66f4668afe30137efb5838f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:10:51.806983 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:10:51.807027 9682 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64e0ccf2e66f4668afe30137efb5838f [term 1 LEADER]: Becoming Leader. State: Replica: 64e0ccf2e66f4668afe30137efb5838f, State: Running, Role: LEADER
I20260812 03:10:51.807320 9682 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64e0ccf2e66f4668afe30137efb5838f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e0ccf2e66f4668afe30137efb5838f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 33667 } }
I20260812 03:10:51.808111 9677 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64e0ccf2e66f4668afe30137efb5838f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:10:51.808511 9682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e0ccf2e66f4668afe30137efb5838f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64e0ccf2e66f4668afe30137efb5838f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e0ccf2e66f4668afe30137efb5838f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 33667 } } }
I20260812 03:10:51.808696 9682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e0ccf2e66f4668afe30137efb5838f [sys.catalog]: This master's current role is: LEADER
I20260812 03:10:51.809142 9683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e0ccf2e66f4668afe30137efb5838f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64e0ccf2e66f4668afe30137efb5838f. Latest consensus state: current_term: 1 leader_uuid: "64e0ccf2e66f4668afe30137efb5838f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e0ccf2e66f4668afe30137efb5838f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 33667 } } }
I20260812 03:10:51.809314 9683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e0ccf2e66f4668afe30137efb5838f [sys.catalog]: This master's current role is: LEADER
I20260812 03:10:51.809356 9687 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:10:51.813413 9687 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:10:51.819739 9687 catalog_manager.cc:1347] Generated new cluster ID: 5c2f2de66f1248d09f1b677eaf93a1cc
I20260812 03:10:51.819852 9687 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:10:51.825429 9687 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:10:51.826588 9687 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:10:51.833031 9687 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64e0ccf2e66f4668afe30137efb5838f: Generated new TSK 0
I20260812 03:10:51.833855 9687 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:10:51.863634 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 9605
2026-08-12T03:10:51Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1633 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:10:51 dist-test-slave-zpfg krb5kdc[9704](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:10:51 dist-test-slave-zpfg krb5kdc[9704](info): set up 2 sockets
Aug 12 03:10:51 dist-test-slave-zpfg krb5kdc[9704](info): commencing operation
krb5kdc: starting...
W20260812 03:10:53.038455 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:10:53 dist-test-slave-zpfg krb5kdc[9704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:10:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:10:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:10:53.187419 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:33995
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:34825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:33995
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:10:53.317721 9717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:10:53.318094 9717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:10:53.318194 9717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:10:53.322620 9717 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:10:53.322747 9717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:10:53.322788 9717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:10:53.322829 9717 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:10:53.322863 9717 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:10:53.329517 9717 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:34825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:33995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:33995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:10:53.330885 9717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:10:53.332012 9717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:10:53.341974 9726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:53.342116 9717 server_base.cc:1034] running on GCE node
W20260812 03:10:53.341974 9723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:53.341992 9724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:53.342924 9717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:10:53.344265 9717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:10:53.345501 9717 hybrid_clock.cc:648] HybridClock initialized: now 1786504253345457 us; error 57 us; skew 500 ppm
Aug 12 03:10:53 dist-test-slave-zpfg krb5kdc[9704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504253, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:10:53.348789 9717 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:10:53.349884 9717 webserver.cc:460] Webserver started at http://127.26.250.190:33011/ using document root <none> and password file <none>
I20260812 03:10:53.350592 9717 fs_manager.cc:359] Metadata directory not provided
I20260812 03:10:53.350667 9717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:10:53.350845 9717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:10:53.352988 9717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "39dbec4451b84572a24d4826c5120016"
format_stamp: "Formatted at 2026-08-12 03:10:53 on dist-test-slave-zpfg"
server_key: "30e312031a8c0ad93255f94ca3f14b7d"
server_key_iv: "9466b21980e80429142fb7f0ddbac666"
server_key_version: "encryptionkey@0"
I20260812 03:10:53.353523 9717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "39dbec4451b84572a24d4826c5120016"
format_stamp: "Formatted at 2026-08-12 03:10:53 on dist-test-slave-zpfg"
server_key: "30e312031a8c0ad93255f94ca3f14b7d"
server_key_iv: "9466b21980e80429142fb7f0ddbac666"
server_key_version: "encryptionkey@0"
I20260812 03:10:53.357885 9717 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 03:10:53.361009 9733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:53.362622 9717 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:10:53.362752 9717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "39dbec4451b84572a24d4826c5120016"
format_stamp: "Formatted at 2026-08-12 03:10:53 on dist-test-slave-zpfg"
server_key: "30e312031a8c0ad93255f94ca3f14b7d"
server_key_iv: "9466b21980e80429142fb7f0ddbac666"
server_key_version: "encryptionkey@0"
I20260812 03:10:53.362854 9717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:10:53.389603 9717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:10:53.393705 9717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:10:53.393905 9717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:10:53.402654 9717 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:33995
I20260812 03:10:53.402994 9790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:33995 every 8 connection(s)
I20260812 03:10:53.403841 9717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:10:53.406769 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 9717
I20260812 03:10:53.406916 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:10:53.406980 9791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:10:53.407245 27626 external_mini_cluster.cc:1428] Setting key 1ac9382930a620f3187fd36689db6157
I20260812 03:10:53.414765 9791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016: Bootstrap starting.
I20260812 03:10:53.417812 9791 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016: Neither blocks nor log segments found. Creating new log.
Aug 12 03:10:53 dist-test-slave-zpfg krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:10:53.418738 9791 log.cc:824] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016: Log is configured to *not* fsync() on all Append() calls
I20260812 03:10:53.421581 9791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016: No bootstrap required, opened a new log
I20260812 03:10:53.425849 9794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:53.408810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44952 (local address 127.26.250.190:33995)
0812 03:10:53.409293 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:53.409311 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:53.409352 (+ 41us) server_negotiation.cc:407] Connection header received
0812 03:10:53.410497 (+ 1145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:53.410520 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:53.410897 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:53.411254 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:53.412531 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:53.413799 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:53.415106 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:53.415828 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:53.416040 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:53.419526 (+ 3486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:53.419532 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:53.419548 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:53.419578 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:53.421883 (+ 2305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:53.422592 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:53.422599 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:53.422605 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:53.422771 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:53.423059 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:53.423074 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:53.423075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:53.423640 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:53.423855 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:53.424434 (+ 579us) server_negotiation.cc:299] Negotiation successful
0812 03:10:53.424777 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":157,"threads_started":1}
I20260812 03:10:53.426680 9791 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39dbec4451b84572a24d4826c5120016" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 33995 } }
I20260812 03:10:53.427002 9791 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:10:53.427065 9791 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39dbec4451b84572a24d4826c5120016, State: Initialized, Role: FOLLOWER
I20260812 03:10:53.427593 9791 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39dbec4451b84572a24d4826c5120016" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 33995 } }
I20260812 03:10:53.427759 9791 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:10:53.427805 9791 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:10:53.427909 9791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:10:53.429147 9791 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39dbec4451b84572a24d4826c5120016" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 33995 } }
I20260812 03:10:53.429560 9791 leader_election.cc:302] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39dbec4451b84572a24d4826c5120016; no voters:
I20260812 03:10:53.429914 9791 leader_election.cc:288] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:10:53.430114 9796 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:10:53.430538 9796 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [term 1 LEADER]: Becoming Leader. State: Replica: 39dbec4451b84572a24d4826c5120016, State: Running, Role: LEADER
I20260812 03:10:53.430943 9796 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39dbec4451b84572a24d4826c5120016" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 33995 } }
I20260812 03:10:53.431295 9791 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:10:53.432770 9797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39dbec4451b84572a24d4826c5120016" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39dbec4451b84572a24d4826c5120016" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 33995 } } }
I20260812 03:10:53.432919 9797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [sys.catalog]: This master's current role is: LEADER
I20260812 03:10:53.433256 9798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39dbec4451b84572a24d4826c5120016. Latest consensus state: current_term: 1 leader_uuid: "39dbec4451b84572a24d4826c5120016" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39dbec4451b84572a24d4826c5120016" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 33995 } } }
I20260812 03:10:53.433364 9798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [sys.catalog]: This master's current role is: LEADER
W20260812 03:10:53.436510 9811 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:10:53.436589 9811 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:10:53.436676 9808 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:10:53.437413 9808 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:10:53.444118 9808 catalog_manager.cc:1347] Generated new cluster ID: 77b1b5ae15974668a2ad4b660461cc9a
I20260812 03:10:53.444222 9808 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:10:53.450770 9808 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:10:53.452164 9808 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:10:53.461314 9808 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016: Generated new TSK 0
I20260812 03:10:53.462185 9808 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.250.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
I20260812 03:10:53.502853 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:0
--local_ip_for_outbound_sockets=127.26.250.129
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:33995
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:34825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:10:53.647405 9817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:10:53.647719 9817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:10:53.647768 9817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:10:53.652172 9817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:10:53.652257 9817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:10:53.652324 9817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:10:53.658202 9817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:34825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:33995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:10:53.659647 9817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:10:53.660848 9817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:10:53.672039 9823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:53.672035 9822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:53.672003 9825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:53.672389 9817 server_base.cc:1034] running on GCE node
I20260812 03:10:53.672865 9817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:10:53.673684 9817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:10:53.674902 9817 hybrid_clock.cc:648] HybridClock initialized: now 1786504253674860 us; error 47 us; skew 500 ppm
Aug 12 03:10:53 dist-test-slave-zpfg krb5kdc[9704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504253, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:10:53.678433 9817 init.cc:377] Logged in from keytab as kudu/127.26.250.129@KRBTEST.COM (short username kudu)
I20260812 03:10:53.679494 9817 webserver.cc:460] Webserver started at http://127.26.250.129:46335/ using document root <none> and password file <none>
I20260812 03:10:53.680192 9817 fs_manager.cc:359] Metadata directory not provided
I20260812 03:10:53.680301 9817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:10:53.680562 9817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:10:53.682936 9817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/data/instance:
uuid: "9269b47f18e146778c6d227fa1467f59"
format_stamp: "Formatted at 2026-08-12 03:10:53 on dist-test-slave-zpfg"
server_key: "0c51f5dae1475c5b8c70802a1f223a08"
server_key_iv: "e9fe2200106f8117943c4dd77328fecb"
server_key_version: "encryptionkey@0"
I20260812 03:10:53.683598 9817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance:
uuid: "9269b47f18e146778c6d227fa1467f59"
format_stamp: "Formatted at 2026-08-12 03:10:53 on dist-test-slave-zpfg"
server_key: "0c51f5dae1475c5b8c70802a1f223a08"
server_key_iv: "e9fe2200106f8117943c4dd77328fecb"
server_key_version: "encryptionkey@0"
I20260812 03:10:53.687840 9817 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:10:53.690618 9832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:53.692129 9817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:10:53.692296 9817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "9269b47f18e146778c6d227fa1467f59"
format_stamp: "Formatted at 2026-08-12 03:10:53 on dist-test-slave-zpfg"
server_key: "0c51f5dae1475c5b8c70802a1f223a08"
server_key_iv: "e9fe2200106f8117943c4dd77328fecb"
server_key_version: "encryptionkey@0"
I20260812 03:10:53.692435 9817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:10:53.716744 9817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:10:53.720611 9817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:10:53.720860 9817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:10:53.721563 9817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:10:53.722579 9817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:10:53.722654 9817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:53.722728 9817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:10:53.722771 9817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:53.733757 9817 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:41525
I20260812 03:10:53.733800 9951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:41525 every 8 connection(s)
I20260812 03:10:53.734843 9817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:10:53.743330 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 9817
I20260812 03:10:53.743561 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance
I20260812 03:10:53.743856 27626 external_mini_cluster.cc:1428] Setting key 267bdff0cb6d7671a65aaa0035081022
Aug 12 03:10:53 dist-test-slave-zpfg krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504253, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:10:53.750279 9794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:53.736919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:51947 (local address 127.26.250.190:33995)
0812 03:10:53.737092 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:53.737097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:53.738141 (+ 1044us) server_negotiation.cc:407] Connection header received
0812 03:10:53.739696 (+ 1555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:53.739699 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:53.739758 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:53.739872 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:53.741157 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:53.741725 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:53.743116 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:53.743365 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:53.743481 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:53.746236 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:53.746240 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:53.746242 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:53.746268 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:53.747899 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:53.748508 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:53.748511 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:53.748513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:53.748607 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:53.749057 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:53.749060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:53.749062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:53.749269 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:53.749396 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:53.749963 (+ 567us) server_negotiation.cc:299] Negotiation successful
0812 03:10:53.750107 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 03:10:53.751243 9954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:53.737427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.129:51947)
0812 03:10:53.737976 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:53.738025 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:53.739466 (+ 1441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:53.740065 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:53.740073 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:53.740462 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:10:53.740993 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:53.741007 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:53.741878 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:53.741881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:53.742957 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:53.742967 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:53.743543 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:53.743548 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:53.743779 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:53.744491 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:10:53.744509 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:10:53.746048 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:10:53.748043 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:53.748051 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:53.748076 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:53.748389 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:53.748711 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:53.748714 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:53.748716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:53.748927 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:53.749396 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:10:53.749405 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:10:53.749694 (+ 289us) client_negotiation.cc:769] Sending connection context
0812 03:10:53.749922 (+ 228us) client_negotiation.cc:240] Negotiation successful
0812 03:10:53.750299 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":116,"threads_started":1}
I20260812 03:10:53.752897 9952 heartbeater.cc:347] Connected to a master server at 127.26.250.190:33995
I20260812 03:10:53.753175 9952 heartbeater.cc:464] Registering TS with master...
I20260812 03:10:53.753778 9952 heartbeater.cc:511] Master 127.26.250.190:33995 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.26.250.130@KRBTEST.COM; defaulting to no policy
I20260812 03:10:53.755774 9751 ts_manager.cc:194] Registered new tserver with Master: 9269b47f18e146778c6d227fa1467f59 (127.26.250.129:41525)
I20260812 03:10:53.757040 9751 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.129@KRBTEST.COM'} at 127.26.250.129:51947
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
I20260812 03:10:53.779870 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:0
--local_ip_for_outbound_sockets=127.26.250.130
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:33995
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:34825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:10:53.903963 9957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:10:53.904345 9957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:10:53.904430 9957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:10:53.908864 9957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:10:53.908989 9957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:10:53.909094 9957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:10:53.914875 9957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:34825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:33995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:10:53.916191 9957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:10:53.917402 9957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:10:53.929912 9962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:53.929926 9964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:53.930065 9957 server_base.cc:1034] running on GCE node
W20260812 03:10:53.930316 9966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:53.930932 9957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:10:53.931891 9957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:10:53.933410 9957 hybrid_clock.cc:648] HybridClock initialized: now 1786504253933354 us; error 76 us; skew 500 ppm
Aug 12 03:10:53 dist-test-slave-zpfg krb5kdc[9704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504253, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:10:53.937147 9957 init.cc:377] Logged in from keytab as kudu/127.26.250.130@KRBTEST.COM (short username kudu)
I20260812 03:10:53.938258 9957 webserver.cc:460] Webserver started at http://127.26.250.130:42481/ using document root <none> and password file <none>
I20260812 03:10:53.939020 9957 fs_manager.cc:359] Metadata directory not provided
I20260812 03:10:53.939100 9957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:10:53.939294 9957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:10:53.941689 9957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/data/instance:
uuid: "119178da7c5a4e07abfbe0e3d159c425"
format_stamp: "Formatted at 2026-08-12 03:10:53 on dist-test-slave-zpfg"
server_key: "e723187ff2b0406dd548ae08285ca77d"
server_key_iv: "c8c821b37645176f9ddca47f84de2a15"
server_key_version: "encryptionkey@0"
I20260812 03:10:53.942305 9957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance:
uuid: "119178da7c5a4e07abfbe0e3d159c425"
format_stamp: "Formatted at 2026-08-12 03:10:53 on dist-test-slave-zpfg"
server_key: "e723187ff2b0406dd548ae08285ca77d"
server_key_iv: "c8c821b37645176f9ddca47f84de2a15"
server_key_version: "encryptionkey@0"
I20260812 03:10:53.946725 9957 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:10:53.949513 9973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:53.951115 9957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:10:53.951265 9957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "119178da7c5a4e07abfbe0e3d159c425"
format_stamp: "Formatted at 2026-08-12 03:10:53 on dist-test-slave-zpfg"
server_key: "e723187ff2b0406dd548ae08285ca77d"
server_key_iv: "c8c821b37645176f9ddca47f84de2a15"
server_key_version: "encryptionkey@0"
I20260812 03:10:53.951397 9957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:10:53.991113 9957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:10:53.995358 9957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:10:53.995604 9957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:10:53.996382 9957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:10:53.997381 9957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:10:53.997460 9957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:53.997527 9957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:10:53.997578 9957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:54.007205 9957 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:35585
I20260812 03:10:54.007230 10090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:35585 every 8 connection(s)
I20260812 03:10:54.008276 9957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:10:54.009779 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 9957
I20260812 03:10:54.009924 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance
I20260812 03:10:54.010211 27626 external_mini_cluster.cc:1428] Setting key cd093255d89a6a47ff62842202768d57
Aug 12 03:10:54 dist-test-slave-zpfg krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504253, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.250.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.131@KRBTEST.COM" created.
I20260812 03:10:54.025303 9794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:54.010565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:45273 (local address 127.26.250.190:33995)
0812 03:10:54.010732 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:54.010737 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:54.011957 (+ 1220us) server_negotiation.cc:407] Connection header received
0812 03:10:54.013386 (+ 1429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:54.013390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:54.013449 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:54.013553 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:54.015020 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.015573 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.016860 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.017097 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.017209 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:54.020558 (+ 3349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:54.020562 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:54.020564 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:54.020593 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:54.022397 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:54.023036 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:54.023039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:54.023040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:54.023126 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:54.023721 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:54.023727 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:54.023733 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:54.023989 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:54.024099 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:54.024919 (+ 820us) server_negotiation.cc:299] Negotiation successful
0812 03:10:54.025066 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 03:10:54.026410 10093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:54.011113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.130:45273)
0812 03:10:54.011786 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:54.011844 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:54.013149 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:54.013724 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:54.013732 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:54.014128 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:10:54.014851 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:54.014866 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.015724 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.015728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:54.016694 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:54.016704 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.017284 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.017289 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:54.017465 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:54.018210 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:10:54.018228 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:10:54.020328 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:10:54.022543 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:54.022550 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:54.022566 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:54.022889 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:54.023312 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:54.023315 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:54.023317 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:54.023497 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:54.024123 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:10:54.024129 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:10:54.024531 (+ 402us) client_negotiation.cc:769] Sending connection context
0812 03:10:54.024817 (+ 286us) client_negotiation.cc:240] Negotiation successful
0812 03:10:54.025331 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":143,"threads_started":1}
I20260812 03:10:54.027958 10091 heartbeater.cc:347] Connected to a master server at 127.26.250.190:33995
I20260812 03:10:54.028381 10091 heartbeater.cc:464] Registering TS with master...
I20260812 03:10:54.029279 10091 heartbeater.cc:511] Master 127.26.250.190:33995 requested a full tablet report, sending...
I20260812 03:10:54.030683 9751 ts_manager.cc:194] Registered new tserver with Master: 119178da7c5a4e07abfbe0e3d159c425 (127.26.250.130:35585)
I20260812 03:10:54.031342 9751 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.130@KRBTEST.COM'} at 127.26.250.130:45273
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
I20260812 03:10:54.046905 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:0
--local_ip_for_outbound_sockets=127.26.250.131
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:33995
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:34825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:10:54.173606 10096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:10:54.173925 10096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:10:54.173971 10096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:10:54.178263 10096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:10:54.178368 10096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:10:54.178432 10096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:10:54.183985 10096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:34825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:33995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:10:54.185238 10096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:10:54.186293 10096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:10:54.196559 10104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:54.196559 10101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:54.196619 10102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:54.197043 10096 server_base.cc:1034] running on GCE node
I20260812 03:10:54.197520 10096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:10:54.198482 10096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:10:54.199673 10096 hybrid_clock.cc:648] HybridClock initialized: now 1786504254199631 us; error 50 us; skew 500 ppm
Aug 12 03:10:54 dist-test-slave-zpfg krb5kdc[9704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504254, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:10:54.203050 10096 init.cc:377] Logged in from keytab as kudu/127.26.250.131@KRBTEST.COM (short username kudu)
I20260812 03:10:54.204231 10096 webserver.cc:460] Webserver started at http://127.26.250.131:35469/ using document root <none> and password file <none>
I20260812 03:10:54.204846 10096 fs_manager.cc:359] Metadata directory not provided
I20260812 03:10:54.204914 10096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:10:54.205149 10096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:10:54.207481 10096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/data/instance:
uuid: "44f89d7ab5a8403cb0ca76caaafc37e4"
format_stamp: "Formatted at 2026-08-12 03:10:54 on dist-test-slave-zpfg"
server_key: "dad9c07676f75f06f34b2c7c11ce4d54"
server_key_iv: "22a77a5e0a9aa97ad3ec7abb825af631"
server_key_version: "encryptionkey@0"
I20260812 03:10:54.208106 10096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance:
uuid: "44f89d7ab5a8403cb0ca76caaafc37e4"
format_stamp: "Formatted at 2026-08-12 03:10:54 on dist-test-slave-zpfg"
server_key: "dad9c07676f75f06f34b2c7c11ce4d54"
server_key_iv: "22a77a5e0a9aa97ad3ec7abb825af631"
server_key_version: "encryptionkey@0"
I20260812 03:10:54.212296 10096 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 03:10:54.215209 10111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:54.216693 10096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:10:54.216958 10096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "44f89d7ab5a8403cb0ca76caaafc37e4"
format_stamp: "Formatted at 2026-08-12 03:10:54 on dist-test-slave-zpfg"
server_key: "dad9c07676f75f06f34b2c7c11ce4d54"
server_key_iv: "22a77a5e0a9aa97ad3ec7abb825af631"
server_key_version: "encryptionkey@0"
I20260812 03:10:54.217093 10096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:10:54.239775 10096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:10:54.243435 10096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:10:54.243666 10096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:10:54.244400 10096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:10:54.245314 10096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:10:54.245362 10096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:54.245427 10096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:10:54.245460 10096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:54.255787 10096 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:36859
I20260812 03:10:54.255851 10227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:36859 every 8 connection(s)
I20260812 03:10:54.256875 10096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:10:54.266305 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 10096
I20260812 03:10:54.266507 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance
I20260812 03:10:54.266855 27626 external_mini_cluster.cc:1428] Setting key f0f3ea5c5cdd752cd96106563be4677e
Aug 12 03:10:54 dist-test-slave-zpfg krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504254, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:10:54.273648 9794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:54.259178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:41363 (local address 127.26.250.190:33995)
0812 03:10:54.259434 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:54.259439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:54.260585 (+ 1146us) server_negotiation.cc:407] Connection header received
0812 03:10:54.262253 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:54.262256 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:54.262316 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:54.262432 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:54.263892 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.264541 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.265906 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.266258 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.266453 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:54.269901 (+ 3448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:54.269905 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:54.269907 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:54.269936 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:54.271434 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:54.272023 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:54.272026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:54.272027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:54.272109 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:54.272460 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:54.272463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:54.272464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:54.272662 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:54.272824 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:54.273401 (+ 577us) server_negotiation.cc:299] Negotiation successful
0812 03:10:54.273520 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260812 03:10:54.274705 10230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:54.259788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.131:41363)
0812 03:10:54.260384 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:54.260437 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:54.261884 (+ 1447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:54.262683 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:54.262692 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:54.263103 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:10:54.263709 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:54.263723 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.264691 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.264694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:54.265739 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:54.265750 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.266588 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.266595 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:54.266786 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:54.267593 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:10:54.267617 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:10:54.269733 (+ 2116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:10:54.271582 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:54.271587 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:54.271602 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:54.271924 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:54.272198 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:54.272201 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:54.272202 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:54.272355 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:54.272884 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:10:54.272890 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:10:54.273165 (+ 275us) client_negotiation.cc:769] Sending connection context
0812 03:10:54.273331 (+ 166us) client_negotiation.cc:240] Negotiation successful
0812 03:10:54.273635 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":146,"threads_started":1}
I20260812 03:10:54.275997 10228 heartbeater.cc:347] Connected to a master server at 127.26.250.190:33995
I20260812 03:10:54.276270 10228 heartbeater.cc:464] Registering TS with master...
I20260812 03:10:54.276827 10228 heartbeater.cc:511] Master 127.26.250.190:33995 requested a full tablet report, sending...
I20260812 03:10:54.278079 9751 ts_manager.cc:194] Registered new tserver with Master: 44f89d7ab5a8403cb0ca76caaafc37e4 (127.26.250.131:36859)
I20260812 03:10:54.278746 9751 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.131@KRBTEST.COM'} at 127.26.250.131:41363
I20260812 03:10:54.281652 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:10:54.292769 9794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:54.283902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44964 (local address 127.26.250.190:33995)
0812 03:10:54.284068 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:54.284073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:54.284323 (+ 250us) server_negotiation.cc:407] Connection header received
0812 03:10:54.284429 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:54.284432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:54.284480 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:54.284550 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:54.285334 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.285940 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.286934 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.287180 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.287291 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:54.288920 (+ 1629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:54.288924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:54.288926 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:54.288953 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:54.290551 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:54.291003 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:54.291006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:54.291007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:54.291097 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:54.291711 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:54.291714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:54.291715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:54.291984 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:54.292097 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:54.292474 (+ 377us) server_negotiation.cc:299] Negotiation successful
0812 03:10:54.292586 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 03:10:54.296707 9751 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44964:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:10:54.300107 9751 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:10:54.316929 10243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:54.310685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59690 (local address 127.26.250.131:36859)
0812 03:10:54.311426 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:54.311440 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:54.311465 (+ 25us) server_negotiation.cc:407] Connection header received
0812 03:10:54.311545 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:54.311550 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:54.311774 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:54.311939 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:10:54.312182 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.313368 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.315193 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.316008 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.316174 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:54.316303 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:54.316642 (+ 339us) server_negotiation.cc:299] Negotiation successful
0812 03:10:54.316757 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":165,"threads_started":1}
I20260812 03:10:54.317022 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:54.309913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:36859 (local address 127.0.0.1:59690)
0812 03:10:54.310992 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:54.311007 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:54.311107 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:54.311919 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:54.311925 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:54.311946 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:10:54.312062 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:54.312068 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.313522 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.313526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:54.315052 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:54.315075 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.316243 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.316246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:54.316393 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:54.316446 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 03:10:54.316585 (+ 139us) client_negotiation.cc:240] Negotiation successful
0812 03:10:54.316846 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1005,"mutex_wait_us":70,"thread_start_us":161,"threads_started":1}
I20260812 03:10:54.317023 10242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:54.310634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:41525 (local address 127.0.0.1:52514)
0812 03:10:54.311629 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:54.311644 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:54.311764 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:54.312206 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:54.312209 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:54.312228 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:10:54.312333 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:54.312340 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.313705 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.313709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:54.314864 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:54.314874 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.316198 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.316202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:54.316472 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:54.316489 (+ 17us) client_negotiation.cc:769] Sending connection context
0812 03:10:54.316585 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 03:10:54.316887 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":932,"mutex_wait_us":161,"thread_start_us":96,"threads_started":1}
I20260812 03:10:54.318025 10239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:54.309806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:35585 (local address 127.0.0.1:41172)
0812 03:10:54.310327 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:54.310421 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:54.310581 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:54.311595 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:54.311602 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:54.311628 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:10:54.311809 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:54.311821 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.313264 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.313271 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:54.315053 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:54.315065 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.317574 (+ 2509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.317578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:54.317688 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:54.317736 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 03:10:54.317810 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 03:10:54.317895 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":95,"thread_start_us":85,"threads_started":1}
I20260812 03:10:54.318041 10244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:54.310991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52514 (local address 127.26.250.129:41525)
0812 03:10:54.311404 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:54.311414 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:54.311766 (+ 352us) server_negotiation.cc:407] Connection header received
0812 03:10:54.311888 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:54.311894 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:54.312080 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:54.312232 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:10:54.312452 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.313564 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.314997 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.315905 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.316058 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:54.316269 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:54.317760 (+ 1491us) server_negotiation.cc:299] Negotiation successful
0812 03:10:54.317883 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":102,"threads_started":1}
I20260812 03:10:54.318500 10240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:54.309934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41172 (local address 127.26.250.130:35585)
0812 03:10:54.310394 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:54.310407 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:54.310765 (+ 358us) server_negotiation.cc:407] Connection header received
0812 03:10:54.310871 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:54.310879 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:54.311130 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:54.311331 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:10:54.311982 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.313100 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.316381 (+ 3281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.317233 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.317368 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:54.317515 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:54.318226 (+ 711us) server_negotiation.cc:299] Negotiation successful
0812 03:10:54.318334 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":94,"threads_started":1}
I20260812 03:10:54.321031 9881 tablet_service.cc:1467] Processing CreateTablet for tablet ed78e734d8ae42d29604265b65243bee (DEFAULT_TABLE table=test-table [id=0a29426fc7794e5da958ec2f18a93cd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:10:54.321118 10162 tablet_service.cc:1467] Processing CreateTablet for tablet ed78e734d8ae42d29604265b65243bee (DEFAULT_TABLE table=test-table [id=0a29426fc7794e5da958ec2f18a93cd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:10:54.321331 10023 tablet_service.cc:1467] Processing CreateTablet for tablet ed78e734d8ae42d29604265b65243bee (DEFAULT_TABLE table=test-table [id=0a29426fc7794e5da958ec2f18a93cd6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:10:54.322206 9881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed78e734d8ae42d29604265b65243bee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:10:54.322243 10023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed78e734d8ae42d29604265b65243bee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:10:54.322261 10162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed78e734d8ae42d29604265b65243bee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:10:54.330168 10245 tablet_bootstrap.cc:492] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59: Bootstrap starting.
I20260812 03:10:54.331089 10246 tablet_bootstrap.cc:492] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425: Bootstrap starting.
I20260812 03:10:54.333276 10245 tablet_bootstrap.cc:654] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59: Neither blocks nor log segments found. Creating new log.
I20260812 03:10:54.333841 10246 tablet_bootstrap.cc:654] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425: Neither blocks nor log segments found. Creating new log.
I20260812 03:10:54.334793 10246 log.cc:824] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425: Log is configured to *not* fsync() on all Append() calls
I20260812 03:10:54.334818 10245 log.cc:824] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59: Log is configured to *not* fsync() on all Append() calls
I20260812 03:10:54.335701 10247 tablet_bootstrap.cc:492] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4: Bootstrap starting.
I20260812 03:10:54.336963 10245 tablet_bootstrap.cc:492] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59: No bootstrap required, opened a new log
I20260812 03:10:54.337189 10245 ts_tablet_manager.cc:1397] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260812 03:10:54.337836 10247 tablet_bootstrap.cc:654] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4: Neither blocks nor log segments found. Creating new log.
I20260812 03:10:54.339197 10247 log.cc:824] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4: Log is configured to *not* fsync() on all Append() calls
I20260812 03:10:54.339919 10246 tablet_bootstrap.cc:492] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425: No bootstrap required, opened a new log
I20260812 03:10:54.340181 10246 ts_tablet_manager.cc:1397] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260812 03:10:54.341431 10247 tablet_bootstrap.cc:492] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4: No bootstrap required, opened a new log
I20260812 03:10:54.341624 10247 ts_tablet_manager.cc:1397] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:10:54.346885 10247 raft_consensus.cc:348] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:54.347234 10247 raft_consensus.cc:374] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:10:54.347318 10247 raft_consensus.cc:729] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44f89d7ab5a8403cb0ca76caaafc37e4, State: Initialized, Role: FOLLOWER
I20260812 03:10:54.347941 10247 consensus_queue.cc:260] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:54.347875 10246 raft_consensus.cc:348] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:54.348142 10246 raft_consensus.cc:374] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:10:54.348192 10246 raft_consensus.cc:729] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 119178da7c5a4e07abfbe0e3d159c425, State: Initialized, Role: FOLLOWER
I20260812 03:10:54.348699 10246 consensus_queue.cc:260] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:54.349190 10247 ts_tablet_manager.cc:1428] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260812 03:10:54.349578 10228 heartbeater.cc:503] Master 127.26.250.190:33995 was elected leader, sending a full tablet report...
I20260812 03:10:54.349764 10246 ts_tablet_manager.cc:1428] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20260812 03:10:54.349614 10245 raft_consensus.cc:348] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:54.350021 10245 raft_consensus.cc:374] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:10:54.350090 10091 heartbeater.cc:503] Master 127.26.250.190:33995 was elected leader, sending a full tablet report...
I20260812 03:10:54.350099 10245 raft_consensus.cc:729] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9269b47f18e146778c6d227fa1467f59, State: Initialized, Role: FOLLOWER
I20260812 03:10:54.350761 10245 consensus_queue.cc:260] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:54.351853 10245 ts_tablet_manager.cc:1428] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59: Time spent starting tablet: real 0.015s user 0.008s sys 0.000s
I20260812 03:10:54.352237 9952 heartbeater.cc:503] Master 127.26.250.190:33995 was elected leader, sending a full tablet report...
I20260812 03:10:54.456506 10251 raft_consensus.cc:482] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:10:54.456774 10251 raft_consensus.cc:504] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:54.458089 10251 leader_election.cc:288] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 119178da7c5a4e07abfbe0e3d159c425 (127.26.250.130:35585), 9269b47f18e146778c6d227fa1467f59 (127.26.250.129:41525)
I20260812 03:10:54.462965 10244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:54.458887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:45971 (local address 127.26.250.129:41525)
0812 03:10:54.459023 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:54.459027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:54.459170 (+ 143us) server_negotiation.cc:407] Connection header received
0812 03:10:54.459325 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:54.459328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:54.459403 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:54.459481 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:10:54.459845 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.460481 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.461685 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.462252 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.462417 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:54.462473 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:54.462730 (+ 257us) server_negotiation.cc:299] Negotiation successful
0812 03:10:54.462818 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:10:54.463308 10230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:54.458313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:35585 (local address 127.26.250.131:51559)
0812 03:10:54.458452 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:54.458469 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:54.458576 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:54.459091 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:54.459097 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:54.459124 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:10:54.459280 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:54.459292 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.460306 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.460312 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:54.461873 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:54.461887 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.462903 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.462907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:54.463023 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:54.463047 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:10:54.463120 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 03:10:54.463202 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":33}
I20260812 03:10:54.463312 10254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:54.458629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:41525 (local address 127.26.250.131:45971)
0812 03:10:54.459067 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:54.459087 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:54.459207 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:54.459518 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:54.459524 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:54.459547 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:10:54.459714 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:54.459723 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.460596 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.460600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:54.461552 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:54.461562 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.462412 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.462417 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:54.462564 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:54.462607 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 03:10:54.462708 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 03:10:54.462815 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":47,"thread_start_us":125,"threads_started":1}
I20260812 03:10:54.463924 10240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:54.458489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:51559 (local address 127.26.250.130:35585)
0812 03:10:54.458658 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:54.458665 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:54.458697 (+ 32us) server_negotiation.cc:407] Connection header received
0812 03:10:54.458768 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:54.458774 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:54.458840 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:54.458938 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:10:54.459419 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.460192 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.462043 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.462632 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.462788 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:54.462842 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:54.463688 (+ 846us) server_negotiation.cc:299] Negotiation successful
0812 03:10:54.463787 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 03:10:54.464182 9903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed78e734d8ae42d29604265b65243bee" candidate_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9269b47f18e146778c6d227fa1467f59" is_pre_election: true
I20260812 03:10:54.464517 9903 raft_consensus.cc:2454] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44f89d7ab5a8403cb0ca76caaafc37e4 in term 0.
I20260812 03:10:54.464603 10044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed78e734d8ae42d29604265b65243bee" candidate_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "119178da7c5a4e07abfbe0e3d159c425" is_pre_election: true
I20260812 03:10:54.464959 10044 raft_consensus.cc:2454] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44f89d7ab5a8403cb0ca76caaafc37e4 in term 0.
I20260812 03:10:54.465085 10114 leader_election.cc:302] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44f89d7ab5a8403cb0ca76caaafc37e4, 9269b47f18e146778c6d227fa1467f59; no voters:
I20260812 03:10:54.465601 10251 raft_consensus.cc:2781] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:10:54.465744 10251 raft_consensus.cc:482] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:10:54.465806 10251 raft_consensus.cc:3037] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:10:54.467665 10251 raft_consensus.cc:504] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:54.468190 10251 leader_election.cc:288] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 [CANDIDATE]: Term 1 election: Requested vote from peers 119178da7c5a4e07abfbe0e3d159c425 (127.26.250.130:35585), 9269b47f18e146778c6d227fa1467f59 (127.26.250.129:41525)
I20260812 03:10:54.468612 10044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed78e734d8ae42d29604265b65243bee" candidate_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "119178da7c5a4e07abfbe0e3d159c425"
I20260812 03:10:54.468735 9903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed78e734d8ae42d29604265b65243bee" candidate_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9269b47f18e146778c6d227fa1467f59"
I20260812 03:10:54.468849 10044 raft_consensus.cc:3037] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:10:54.468904 9903 raft_consensus.cc:3037] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:10:54.470072 9903 raft_consensus.cc:2454] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44f89d7ab5a8403cb0ca76caaafc37e4 in term 1.
I20260812 03:10:54.470429 10114 leader_election.cc:302] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44f89d7ab5a8403cb0ca76caaafc37e4, 9269b47f18e146778c6d227fa1467f59; no voters:
I20260812 03:10:54.470664 10251 raft_consensus.cc:2781] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:10:54.470624 10044 raft_consensus.cc:2454] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44f89d7ab5a8403cb0ca76caaafc37e4 in term 1.
I20260812 03:10:54.470930 10251 raft_consensus.cc:686] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 1 LEADER]: Becoming Leader. State: Replica: 44f89d7ab5a8403cb0ca76caaafc37e4, State: Running, Role: LEADER
I20260812 03:10:54.471335 10251 consensus_queue.cc:237] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:54.475029 9751 catalog_manager.cc:5495] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 44f89d7ab5a8403cb0ca76caaafc37e4 (127.26.250.131). New cstate: current_term: 1 leader_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } health_report { overall_health: UNKNOWN } } }
W20260812 03:10:54.486485 9953 tablet.cc:2369] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:10:54.508082 10229 tablet.cc:2369] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:10:54.509678 10092 tablet.cc:2369] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:10:54.514473 10243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:54.509540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59706 (local address 127.26.250.131:36859)
0812 03:10:54.509680 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:54.509685 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:54.509703 (+ 18us) server_negotiation.cc:407] Connection header received
0812 03:10:54.509944 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:54.509950 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:54.510041 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:54.510162 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:10:54.510533 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.511462 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.512373 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:54.512831 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:54.512954 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:54.513229 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:10:54.513866 (+ 637us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:10:54.514012 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:54.514123 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 03:10:54.514207 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 03:10:54.521226 10044 raft_consensus.cc:1261] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425 [term 1 FOLLOWER]: Refusing update from remote peer 44f89d7ab5a8403cb0ca76caaafc37e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:10:54.521226 9903 raft_consensus.cc:1261] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Refusing update from remote peer 44f89d7ab5a8403cb0ca76caaafc37e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:10:54.522658 10255 consensus_queue.cc:1035] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260812 03:10:54.522945 10251 consensus_queue.cc:1035] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:10:54.530640 10258 mvcc.cc:205] Tried to move back new op lower bound from 7317521426509791232 to 7317521426317611008. Current Snapshot: MvccSnapshot[applied={T|T < 7317521426509791232}]
I20260812 03:10:54.534148 10260 mvcc.cc:205] Tried to move back new op lower bound from 7317521426509791232 to 7317521426317611008. Current Snapshot: MvccSnapshot[applied={T|T < 7317521426509791232}]
I20260812 03:10:54.543393 9750 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:10:54.543593 9750 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:10:54.546450 9750 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016: Sending DeleteTablet for 3 replicas of tablet ed78e734d8ae42d29604265b65243bee
I20260812 03:10:54.547420 10023 tablet_service.cc:1514] Processing DeleteTablet for tablet ed78e734d8ae42d29604265b65243bee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:10:54 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:41172
I20260812 03:10:54.547536 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 9817
I20260812 03:10:54.548552 10162 tablet_service.cc:1514] Processing DeleteTablet for tablet ed78e734d8ae42d29604265b65243bee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:10:54 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:59690
I20260812 03:10:54.554644 10271 tablet_replica.cc:330] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4: stopping tablet replica
I20260812 03:10:54.555050 10269 tablet_replica.cc:330] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425: stopping tablet replica
I20260812 03:10:54.555681 10271 raft_consensus.cc:2229] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:10:54.556100 10269 raft_consensus.cc:2229] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:10:54.556900 10271 raft_consensus.cc:2258] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:10:54.557224 10269 raft_consensus.cc:2258] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 03:10:54.557888 9737 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.250.129:41525 (error 108)
I20260812 03:10:54.559353 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 9957
I20260812 03:10:54.559414 10239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:54.559076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:41525 (local address 127.0.0.1:52518)
0812 03:10:54.559233 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:54.559318 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.129:41525: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 03:10:54.560187 9737 catalog_manager.cc:4584] TS 9269b47f18e146778c6d227fa1467f59 (127.26.250.129:41525): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ed78e734d8ae42d29604265b65243bee: Network error: Client connection negotiation failed: client connection to 127.26.250.129:41525: connect: Connection refused (error 111)
I20260812 03:10:54.576377 10271 ts_tablet_manager.cc:1905] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 03:10:54.577092 10239 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:10:54.576319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:35585 (local address 127.0.0.1:41182)
0812 03:10:54.576547 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:54.576564 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:54.576670 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:54.576941 (+ 271us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.130:35585: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":134}
I20260812 03:10:54.577801 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 10096
I20260812 03:10:54.587159 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 9717
I20260812 03:10:54.597105 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:33995
--webserver_interface=127.26.250.190
--webserver_port=33011
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:34825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:33995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:10:54.719547 10273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:10:54.719884 10273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:10:54.719952 10273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:10:54.724154 10273 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:10:54.724223 10273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:10:54.724243 10273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:10:54.724260 10273 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:10:54.724277 10273 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:10:54.730069 10273 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:34825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:33995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:33995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=33011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:10:54.731407 10273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:10:54.732528 10273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:10:54.748921 10280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:54.748943 10282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:54.748921 10279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:54.749346 10273 server_base.cc:1034] running on GCE node
I20260812 03:10:54.750087 10273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:10:54.751587 10273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:10:54.752808 10273 hybrid_clock.cc:648] HybridClock initialized: now 1786504254752776 us; error 50 us; skew 500 ppm
Aug 12 03:10:54 dist-test-slave-zpfg krb5kdc[9704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504254, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:10:54.756240 10273 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:10:54.757452 10273 webserver.cc:460] Webserver started at http://127.26.250.190:33011/ using document root <none> and password file <none>
I20260812 03:10:54.758162 10273 fs_manager.cc:359] Metadata directory not provided
I20260812 03:10:54.758304 10273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:10:54.763069 10273 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:10:54.765748 10289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:54.767385 10273 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:10:54.767563 10273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "39dbec4451b84572a24d4826c5120016"
format_stamp: "Formatted at 2026-08-12 03:10:53 on dist-test-slave-zpfg"
server_key: "30e312031a8c0ad93255f94ca3f14b7d"
server_key_iv: "9466b21980e80429142fb7f0ddbac666"
server_key_version: "encryptionkey@0"
I20260812 03:10:54.768318 10273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:10:54.801859 10273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:10:54.805516 10273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:10:54.805748 10273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:10:54.813786 10273 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:33995
I20260812 03:10:54.814116 10343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:33995 every 8 connection(s)
I20260812 03:10:54.815114 10273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:10:54.815661 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 10273
I20260812 03:10:54.816172 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:41525
--local_ip_for_outbound_sockets=127.26.250.129
--tserver_master_addrs=127.26.250.190:33995
--webserver_port=46335
--webserver_interface=127.26.250.129
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:34825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:10:54.820756 10344 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:10:54.822818 10344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016: Bootstrap starting.
I20260812 03:10:54.841948 10344 log.cc:824] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016: Log is configured to *not* fsync() on all Append() calls
I20260812 03:10:54.851060 10344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:10:54.851468 10344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016: Bootstrap complete.
I20260812 03:10:54.855929 10344 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39dbec4451b84572a24d4826c5120016" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 33995 } }
I20260812 03:10:54.856405 10344 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39dbec4451b84572a24d4826c5120016, State: Initialized, Role: FOLLOWER
I20260812 03:10:54.856978 10344 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39dbec4451b84572a24d4826c5120016" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 33995 } }
I20260812 03:10:54.857133 10344 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:10:54.857235 10344 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:10:54.857359 10344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:10:54.859511 10344 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39dbec4451b84572a24d4826c5120016" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 33995 } }
I20260812 03:10:54.859879 10344 leader_election.cc:302] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39dbec4451b84572a24d4826c5120016; no voters:
I20260812 03:10:54.860241 10344 leader_election.cc:288] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 03:10:54.860395 10348 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:10:54.860669 10348 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [term 2 LEADER]: Becoming Leader. State: Replica: 39dbec4451b84572a24d4826c5120016, State: Running, Role: LEADER
I20260812 03:10:54.861105 10348 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39dbec4451b84572a24d4826c5120016" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 33995 } }
I20260812 03:10:54.861598 10344 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:10:54.862643 10349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "39dbec4451b84572a24d4826c5120016" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39dbec4451b84572a24d4826c5120016" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 33995 } } }
I20260812 03:10:54.862798 10349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [sys.catalog]: This master's current role is: LEADER
I20260812 03:10:54.863078 10350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39dbec4451b84572a24d4826c5120016. Latest consensus state: current_term: 2 leader_uuid: "39dbec4451b84572a24d4826c5120016" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39dbec4451b84572a24d4826c5120016" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 33995 } } }
I20260812 03:10:54.863157 10350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [sys.catalog]: This master's current role is: LEADER
I20260812 03:10:54.867209 10364 catalog_manager.cc:1259] Loaded cluster ID: 77b1b5ae15974668a2ad4b660461cc9a
I20260812 03:10:54.867278 10364 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016: loading cluster ID for follower catalog manager: success
W20260812 03:10:54.868335 10364 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260812 03:10:54.868419 10364 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260812 03:10:54.868503 10357 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:10:54.871311 10357 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:10:54.871848 10357 catalog_manager.cc:1259] Loaded cluster ID: 77b1b5ae15974668a2ad4b660461cc9a
I20260812 03:10:54.871904 10357 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:10:54.873126 10357 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 03:10:54.878070 10357 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 03:10:54.884016 10357 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:10:54.884887 10357 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016: Loaded TSK: 0
I20260812 03:10:54.886117 10357 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:10:54.940759 10346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:10:54.941095 10346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:10:54.941177 10346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:10:54.945462 10346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:10:54.945573 10346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:10:54.945672 10346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:10:54.951297 10346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:34825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:41525
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=46335
--tserver_master_addrs=127.26.250.190:33995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:10:54.952450 10346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:10:54.953523 10346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:10:54.964838 10375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:54.964838 10372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:54.965022 10346 server_base.cc:1034] running on GCE node
W20260812 03:10:54.965132 10373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:54.965724 10346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:10:54.966616 10346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:10:54.967846 10346 hybrid_clock.cc:648] HybridClock initialized: now 1786504254967811 us; error 60 us; skew 500 ppm
Aug 12 03:10:54 dist-test-slave-zpfg krb5kdc[9704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504254, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:10:54.971719 10346 init.cc:377] Logged in from keytab as kudu/127.26.250.129@KRBTEST.COM (short username kudu)
I20260812 03:10:54.972942 10346 webserver.cc:460] Webserver started at http://127.26.250.129:46335/ using document root <none> and password file <none>
I20260812 03:10:54.973685 10346 fs_manager.cc:359] Metadata directory not provided
I20260812 03:10:54.973799 10346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:10:54.979300 10346 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 03:10:54.982299 10382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:54.984058 10346 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 03:10:54.984193 10346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "9269b47f18e146778c6d227fa1467f59"
format_stamp: "Formatted at 2026-08-12 03:10:53 on dist-test-slave-zpfg"
server_key: "0c51f5dae1475c5b8c70802a1f223a08"
server_key_iv: "e9fe2200106f8117943c4dd77328fecb"
server_key_version: "encryptionkey@0"
I20260812 03:10:54.984948 10346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:10:55.019747 10346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:10:55.023897 10346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:10:55.024149 10346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:10:55.025023 10346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:10:55.026764 10389 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 03:10:55.031577 10346 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 03:10:55.031672 10346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260812 03:10:55.031744 10346 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 03:10:55.034020 10346 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 03:10:55.034089 10346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 03:10:55.034487 10389 tablet_bootstrap.cc:492] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59: Bootstrap starting.
I20260812 03:10:55.045487 10346 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:41525
I20260812 03:10:55.045589 10502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:41525 every 8 connection(s)
I20260812 03:10:55.046640 10346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:10:55.055352 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 10346
I20260812 03:10:55.055577 10389 log.cc:824] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59: Log is configured to *not* fsync() on all Append() calls
I20260812 03:10:55.055951 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:35585
--local_ip_for_outbound_sockets=127.26.250.130
--tserver_master_addrs=127.26.250.190:33995
--webserver_port=42481
--webserver_interface=127.26.250.130
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:34825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:10:55 dist-test-slave-zpfg krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504254, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:10:55.065887 10389 tablet_bootstrap.cc:492] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:10:55.066434 10389 tablet_bootstrap.cc:492] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59: Bootstrap complete.
I20260812 03:10:55.067432 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:55.048793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:56471 (local address 127.26.250.190:33995)
0812 03:10:55.049219 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:55.049239 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:55.049644 (+ 405us) server_negotiation.cc:407] Connection header received
0812 03:10:55.051270 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:55.051282 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:55.051615 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:55.051897 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:55.053541 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.054631 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.056387 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.056939 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.057119 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:55.060646 (+ 3527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:55.060653 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:55.060669 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:55.060704 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:55.063496 (+ 2792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:55.064266 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:55.064274 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:55.064280 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:55.064458 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:55.064856 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:55.064861 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:55.064863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:55.065282 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:55.065446 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:55.066031 (+ 585us) server_negotiation.cc:299] Negotiation successful
0812 03:10:55.066369 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":120,"threads_started":1}
I20260812 03:10:55.067457 10506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:55.048783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.129:56471)
0812 03:10:55.049422 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:55.049490 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:55.050751 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:55.052048 (+ 1297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:55.052057 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:55.052571 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:10:55.053357 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:55.053378 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.054799 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.054803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:55.056219 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:55.056230 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.057115 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.057123 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:55.057415 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:55.058208 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:10:55.058237 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:10:55.060389 (+ 2152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:10:55.063638 (+ 3249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:55.063647 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:55.063662 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:55.064108 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:55.064576 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:55.064580 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:55.064583 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:55.064745 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:55.065414 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:10:55.065421 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:10:55.065697 (+ 276us) client_negotiation.cc:769] Sending connection context
0812 03:10:55.065904 (+ 207us) client_negotiation.cc:240] Negotiation successful
0812 03:10:55.066328 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":175,"threads_started":1}
I20260812 03:10:55.068214 10389 ts_tablet_manager.cc:1397] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59: Time spent bootstrapping tablet: real 0.034s user 0.016s sys 0.014s
I20260812 03:10:55.069952 10503 heartbeater.cc:347] Connected to a master server at 127.26.250.190:33995
I20260812 03:10:55.070226 10503 heartbeater.cc:464] Registering TS with master...
I20260812 03:10:55.070819 10503 heartbeater.cc:511] Master 127.26.250.190:33995 requested a full tablet report, sending...
I20260812 03:10:55.072206 10389 raft_consensus.cc:348] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:55.072885 10389 raft_consensus.cc:729] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9269b47f18e146778c6d227fa1467f59, State: Initialized, Role: FOLLOWER
I20260812 03:10:55.073463 10389 consensus_queue.cc:260] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:55.073735 10305 ts_manager.cc:194] Registered new tserver with Master: 9269b47f18e146778c6d227fa1467f59 (127.26.250.129:41525)
I20260812 03:10:55.074570 10389 ts_tablet_manager.cc:1428] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59: Time spent starting tablet: real 0.006s user 0.007s sys 0.001s
I20260812 03:10:55.077755 10305 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.129@KRBTEST.COM'} at 127.26.250.129:56471
I20260812 03:10:55.080685 10503 heartbeater.cc:503] Master 127.26.250.190:33995 was elected leader, sending a full tablet report...
Aug 12 03:10:55 dist-test-slave-zpfg krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504254, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for kudu/127.26.250.129@KRBTEST.COM
I20260812 03:10:55.089448 10510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:55.077012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:41525 (local address 127.0.0.1:52524)
0812 03:10:55.077764 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:55.077817 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:55.077952 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:55.079123 (+ 1171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:55.079130 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:55.079532 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:10:55.079654 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:55.079661 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.080732 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.080737 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:55.081711 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:55.081727 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.082736 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.082739 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:55.082829 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:55.083478 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:10:55.083491 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:10:55.085157 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:10:55.087536 (+ 2379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:55.087546 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:55.087553 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:55.087867 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:55.088218 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:55.088221 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:55.088223 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:55.088321 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:55.088791 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:10:55.088796 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:10:55.088925 (+ 129us) client_negotiation.cc:769] Sending connection context
0812 03:10:55.089069 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 03:10:55.089264 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":130,"threads_started":1}
I20260812 03:10:55.089445 10511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:55.077274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52524 (local address 127.26.250.129:41525)
0812 03:10:55.078587 (+ 1313us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:55.078599 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:55.078627 (+ 28us) server_negotiation.cc:407] Connection header received
0812 03:10:55.078708 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:55.078713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:55.078954 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:55.079123 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:55.079791 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.080590 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.081862 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.082475 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.082585 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:55.085341 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:55.085351 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:55.085363 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:55.085387 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:55.087411 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:55.088012 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:55.088018 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:55.088019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:55.088107 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:55.088422 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:55.088424 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:55.088425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:55.088668 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:55.088803 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:55.089147 (+ 344us) server_negotiation.cc:299] Negotiation successful
0812 03:10:55.089298 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1202,"thread_start_us":118,"threads_started":1}
I20260812 03:10:55.091003 10435 tablet_service.cc:1514] Processing DeleteTablet for tablet ed78e734d8ae42d29604265b65243bee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:10:54 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:52524
I20260812 03:10:55.092048 10512 tablet_replica.cc:330] stopping tablet replica
I20260812 03:10:55.092374 10512 raft_consensus.cc:2229] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:10:55.092643 10512 raft_consensus.cc:2258] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:10:55.093689 10512 ts_tablet_manager.cc:1905] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:10:55.097635 10512 ts_tablet_manager.cc:1918] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:10:55.097740 10512 log.cc:1196] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/wal/wals/ed78e734d8ae42d29604265b65243bee
I20260812 03:10:55.098116 10512 ts_tablet_manager.cc:1939] T ed78e734d8ae42d29604265b65243bee P 9269b47f18e146778c6d227fa1467f59: Deleting consensus metadata
I20260812 03:10:55.100075 10292 catalog_manager.cc:4841] TS 9269b47f18e146778c6d227fa1467f59 (127.26.250.129:41525): tablet ed78e734d8ae42d29604265b65243bee (table test-table [id=0a29426fc7794e5da958ec2f18a93cd6]) successfully deleted
W20260812 03:10:55.188068 10508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:10:55.188390 10508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:10:55.188509 10508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:10:55.193356 10508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:10:55.193475 10508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:10:55.193589 10508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:10:55.199270 10508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:34825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:35585
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=42481
--tserver_master_addrs=127.26.250.190:33995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:10:55.200460 10508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:10:55.201546 10508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:10:55.212515 10519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:55.212545 10508 server_base.cc:1034] running on GCE node
W20260812 03:10:55.212481 10521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:55.212545 10517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:55.213317 10508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:10:55.214062 10508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:10:55.215334 10508 hybrid_clock.cc:648] HybridClock initialized: now 1786504255215292 us; error 54 us; skew 500 ppm
Aug 12 03:10:55 dist-test-slave-zpfg krb5kdc[9704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504255, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:10:55.218561 10508 init.cc:377] Logged in from keytab as kudu/127.26.250.130@KRBTEST.COM (short username kudu)
I20260812 03:10:55.219552 10508 webserver.cc:460] Webserver started at http://127.26.250.130:42481/ using document root <none> and password file <none>
I20260812 03:10:55.220148 10508 fs_manager.cc:359] Metadata directory not provided
I20260812 03:10:55.220278 10508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:10:55.224581 10508 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:10:55.227207 10528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:55.228796 10508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:10:55.228945 10508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "119178da7c5a4e07abfbe0e3d159c425"
format_stamp: "Formatted at 2026-08-12 03:10:53 on dist-test-slave-zpfg"
server_key: "e723187ff2b0406dd548ae08285ca77d"
server_key_iv: "c8c821b37645176f9ddca47f84de2a15"
server_key_version: "encryptionkey@0"
I20260812 03:10:55.229635 10508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:10:55.243393 10508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:10:55.246814 10508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:10:55.246981 10508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:10:55.247561 10508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:10:55.248699 10535 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 03:10:55.253202 10508 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 03:10:55.253300 10508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260812 03:10:55.253407 10508 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 03:10:55.255589 10508 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 03:10:55.255690 10508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260812 03:10:55.255972 10535 tablet_bootstrap.cc:492] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425: Bootstrap starting.
I20260812 03:10:55.262723 10535 log.cc:824] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425: Log is configured to *not* fsync() on all Append() calls
I20260812 03:10:55.267705 10508 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:35585
I20260812 03:10:55.267776 10649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:35585 every 8 connection(s)
I20260812 03:10:55.268661 10508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:10:55.271734 10535 tablet_bootstrap.cc:492] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:10:55.272207 10535 tablet_bootstrap.cc:492] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425: Bootstrap complete.
I20260812 03:10:55.272611 10535 ts_tablet_manager.cc:1397] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425: Time spent bootstrapping tablet: real 0.017s user 0.014s sys 0.000s
I20260812 03:10:55.275650 10535 raft_consensus.cc:348] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:55.276201 10535 raft_consensus.cc:729] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 119178da7c5a4e07abfbe0e3d159c425, State: Initialized, Role: FOLLOWER
I20260812 03:10:55.276713 10535 consensus_queue.cc:260] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:55.277612 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 10508
I20260812 03:10:55.277884 10535 ts_tablet_manager.cc:1428] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260812 03:10:55.278138 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:36859
--local_ip_for_outbound_sockets=127.26.250.131
--tserver_master_addrs=127.26.250.190:33995
--webserver_port=35469
--webserver_interface=127.26.250.131
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:34825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:10:55 dist-test-slave-zpfg krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504255, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:10:55.285122 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:55.270685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:41595 (local address 127.26.250.190:33995)
0812 03:10:55.270860 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:55.270865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:55.271560 (+ 695us) server_negotiation.cc:407] Connection header received
0812 03:10:55.272899 (+ 1339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:55.272903 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:55.272996 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:55.273113 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:55.274608 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.275146 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.276755 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.277115 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.277264 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:55.280775 (+ 3511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:55.280778 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:55.280780 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:55.280810 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:55.282516 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:55.283178 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:55.283184 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:55.283189 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:55.283293 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:55.283719 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:55.283725 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:55.283730 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:55.284017 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:55.284129 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:55.284675 (+ 546us) server_negotiation.cc:299] Negotiation successful
0812 03:10:55.284858 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 03:10:55.285972 10652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:55.270854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.130:41595)
0812 03:10:55.271374 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:55.271430 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:55.272617 (+ 1187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:55.273266 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:55.273275 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:55.273748 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:10:55.274425 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:55.274444 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.275314 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.275317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:55.276615 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:55.276625 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.277259 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.277263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:55.277449 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:55.278737 (+ 1288us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:10:55.278754 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:10:55.280515 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:10:55.282646 (+ 2131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:55.282651 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:55.282666 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:55.283064 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:55.283462 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:55.283464 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:55.283466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:55.283617 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:55.284126 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:10:55.284131 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:10:55.284401 (+ 270us) client_negotiation.cc:769] Sending connection context
0812 03:10:55.284572 (+ 171us) client_negotiation.cc:240] Negotiation successful
0812 03:10:55.284952 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":141,"threads_started":1}
I20260812 03:10:55.287464 10650 heartbeater.cc:347] Connected to a master server at 127.26.250.190:33995
I20260812 03:10:55.287727 10650 heartbeater.cc:464] Registering TS with master...
I20260812 03:10:55.288286 10650 heartbeater.cc:511] Master 127.26.250.190:33995 requested a full tablet report, sending...
I20260812 03:10:55.290158 10305 ts_manager.cc:194] Registered new tserver with Master: 119178da7c5a4e07abfbe0e3d159c425 (127.26.250.130:35585)
I20260812 03:10:55.291505 10305 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.130@KRBTEST.COM'} at 127.26.250.130:41595
I20260812 03:10:55.294158 10650 heartbeater.cc:503] Master 127.26.250.190:33995 was elected leader, sending a full tablet report...
Aug 12 03:10:55 dist-test-slave-zpfg krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504254, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for kudu/127.26.250.130@KRBTEST.COM
I20260812 03:10:55.307554 10510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:55.291327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:35585 (local address 127.0.0.1:41194)
0812 03:10:55.291802 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:55.291818 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:55.291906 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:55.292474 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:55.292480 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:55.292684 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:10:55.292796 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:55.292808 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.294504 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.294508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:55.295306 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:55.295318 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.296205 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.296208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:55.296287 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:55.301591 (+ 5304us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:10:55.301606 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:10:55.303264 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:10:55.305725 (+ 2461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:55.305728 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:55.305729 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:55.305950 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:55.306325 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:55.306327 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:55.306329 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:55.306505 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:55.307073 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:10:55.307075 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:10:55.307174 (+ 99us) client_negotiation.cc:769] Sending connection context
0812 03:10:55.307262 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 03:10:55.307418 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391}
I20260812 03:10:55.307760 10655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:55.291418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41194 (local address 127.26.250.130:35585)
0812 03:10:55.291861 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:55.291878 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:55.291912 (+ 34us) server_negotiation.cc:407] Connection header received
0812 03:10:55.292009 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:55.292020 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:55.292306 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:55.292492 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:55.292929 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.294283 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.295450 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.296075 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.296204 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:55.303473 (+ 7269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:55.303491 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:55.303511 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:55.303545 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:55.305591 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:55.306078 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:55.306085 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:55.306087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:55.306196 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:55.306606 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:55.306610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:55.306611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:55.306964 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:55.307101 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:55.307374 (+ 273us) server_negotiation.cc:299] Negotiation successful
0812 03:10:55.307570 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":131,"threads_started":1}
I20260812 03:10:55.308931 10580 tablet_service.cc:1514] Processing DeleteTablet for tablet ed78e734d8ae42d29604265b65243bee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:10:54 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:41194
I20260812 03:10:55.310052 10656 tablet_replica.cc:330] stopping tablet replica
I20260812 03:10:55.310447 10656 raft_consensus.cc:2229] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:10:55.310730 10656 raft_consensus.cc:2258] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:10:55.311861 10656 ts_tablet_manager.cc:1905] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:10:55.316015 10656 ts_tablet_manager.cc:1918] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:10:55.316119 10656 log.cc:1196] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/wal/wals/ed78e734d8ae42d29604265b65243bee
I20260812 03:10:55.316463 10656 ts_tablet_manager.cc:1939] T ed78e734d8ae42d29604265b65243bee P 119178da7c5a4e07abfbe0e3d159c425: Deleting consensus metadata
I20260812 03:10:55.318173 10292 catalog_manager.cc:4841] TS 119178da7c5a4e07abfbe0e3d159c425 (127.26.250.130:35585): tablet ed78e734d8ae42d29604265b65243bee (table test-table [id=0a29426fc7794e5da958ec2f18a93cd6]) successfully deleted
W20260812 03:10:55.409674 10654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:10:55.409972 10654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:10:55.410054 10654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:10:55.414534 10654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:10:55.414623 10654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:10:55.414716 10654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:10:55.420310 10654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:34825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:36859
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=35469
--tserver_master_addrs=127.26.250.190:33995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:10:55.421486 10654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:10:55.422637 10654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:10:55.433212 10664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:55.433329 10661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:55.433180 10662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:55.433666 10654 server_base.cc:1034] running on GCE node
I20260812 03:10:55.434203 10654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:10:55.434932 10654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:10:55.436267 10654 hybrid_clock.cc:648] HybridClock initialized: now 1786504255436139 us; error 55 us; skew 500 ppm
Aug 12 03:10:55 dist-test-slave-zpfg krb5kdc[9704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504255, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:10:55.439766 10654 init.cc:377] Logged in from keytab as kudu/127.26.250.131@KRBTEST.COM (short username kudu)
I20260812 03:10:55.440864 10654 webserver.cc:460] Webserver started at http://127.26.250.131:35469/ using document root <none> and password file <none>
I20260812 03:10:55.441646 10654 fs_manager.cc:359] Metadata directory not provided
I20260812 03:10:55.441783 10654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:10:55.446277 10654 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:10:55.448957 10672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:55.450579 10654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:10:55.450753 10654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "44f89d7ab5a8403cb0ca76caaafc37e4"
format_stamp: "Formatted at 2026-08-12 03:10:54 on dist-test-slave-zpfg"
server_key: "dad9c07676f75f06f34b2c7c11ce4d54"
server_key_iv: "22a77a5e0a9aa97ad3ec7abb825af631"
server_key_version: "encryptionkey@0"
I20260812 03:10:55.451395 10654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:10:55.486088 10654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:10:55.489877 10654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:10:55.490123 10654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:10:55.490857 10654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:10:55.492051 10680 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 03:10:55.497303 10654 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 03:10:55.497423 10654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260812 03:10:55.497524 10654 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 03:10:55.499657 10654 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 03:10:55.499748 10654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260812 03:10:55.500057 10680 tablet_bootstrap.cc:492] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4: Bootstrap starting.
I20260812 03:10:55.506503 10680 log.cc:824] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4: Log is configured to *not* fsync() on all Append() calls
I20260812 03:10:55.512213 10654 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:36859
I20260812 03:10:55.512288 10792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:36859 every 8 connection(s)
I20260812 03:10:55.513226 10654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:10:55.515427 10680 tablet_bootstrap.cc:492] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:10:55.515935 10680 tablet_bootstrap.cc:492] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4: Bootstrap complete.
I20260812 03:10:55.516355 10680 ts_tablet_manager.cc:1397] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4: Time spent bootstrapping tablet: real 0.017s user 0.014s sys 0.000s
I20260812 03:10:55.519104 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 10654
I20260812 03:10:55.519562 10680 raft_consensus.cc:348] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:55.520134 10680 raft_consensus.cc:729] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44f89d7ab5a8403cb0ca76caaafc37e4, State: Initialized, Role: FOLLOWER
I20260812 03:10:55.520661 10680 consensus_queue.cc:260] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:55.521680 10680 ts_tablet_manager.cc:1428] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
Aug 12 03:10:55 dist-test-slave-zpfg krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504255, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:10:55.530120 10797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:55.519901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44972 (local address 127.26.250.190:33995)
0812 03:10:55.520145 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:55.520151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:55.520462 (+ 311us) server_negotiation.cc:407] Connection header received
0812 03:10:55.520627 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:55.520630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:55.520682 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:55.520779 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:55.521668 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.522497 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.523399 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.523648 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.523759 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:55.525357 (+ 1598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:55.525361 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:55.525364 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:55.525391 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:55.527488 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:55.528312 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:55.528316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:55.528318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:55.528456 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:55.528886 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:55.528889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:55.528891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:55.529312 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:55.529539 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:55.529759 (+ 220us) server_negotiation.cc:299] Negotiation successful
0812 03:10:55.529906 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":172,"thread_start_us":69,"threads_started":1}
I20260812 03:10:55.530395 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:55.515395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:39023 (local address 127.26.250.190:33995)
0812 03:10:55.515572 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:55.515577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:55.516338 (+ 761us) server_negotiation.cc:407] Connection header received
0812 03:10:55.517602 (+ 1264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:55.517605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:55.517668 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:55.517777 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:55.519428 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.520216 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.521781 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.522163 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.522266 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:55.525519 (+ 3253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:55.525525 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:55.525529 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:55.525568 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:55.527501 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:55.528299 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:55.528304 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:55.528306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:55.528457 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:55.528985 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:55.528988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:55.528990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:55.529312 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:55.530032 (+ 720us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:55.530116 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 03:10:55.530210 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":70}
I20260812 03:10:55.531590 10796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:55.515531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.131:39023)
0812 03:10:55.516117 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:55.516196 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:55.517300 (+ 1104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:55.517954 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:55.517963 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:55.518505 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:10:55.519159 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:55.519178 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.520394 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.520397 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:55.521657 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:55.521669 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.522280 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.522283 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:55.522546 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:55.523593 (+ 1047us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:10:55.523610 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:10:55.525360 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:10:55.527693 (+ 2333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:55.527703 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:55.527723 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:55.528131 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:55.528587 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:55.528590 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:55.528592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:55.528876 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:55.529462 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:10:55.529471 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:10:55.529835 (+ 364us) client_negotiation.cc:769] Sending connection context
0812 03:10:55.530042 (+ 207us) client_negotiation.cc:240] Negotiation successful
0812 03:10:55.530442 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":158,"threads_started":1}
I20260812 03:10:55.532747 10793 heartbeater.cc:347] Connected to a master server at 127.26.250.190:33995
I20260812 03:10:55.533037 10793 heartbeater.cc:464] Registering TS with master...
I20260812 03:10:55.533623 10793 heartbeater.cc:511] Master 127.26.250.190:33995 requested a full tablet report, sending...
I20260812 03:10:55.535444 10305 ts_manager.cc:194] Registered new tserver with Master: 44f89d7ab5a8403cb0ca76caaafc37e4 (127.26.250.131:36859)
I20260812 03:10:55.536684 10305 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.131@KRBTEST.COM'} at 127.26.250.131:39023
I20260812 03:10:55.538801 10793 heartbeater.cc:503] Master 127.26.250.190:33995 was elected leader, sending a full tablet report...
Aug 12 03:10:55 dist-test-slave-zpfg krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504254, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for kudu/127.26.250.131@KRBTEST.COM
I20260812 03:10:55.544672 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:10:55.546489 10510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:55.536314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:36859 (local address 127.0.0.1:59722)
0812 03:10:55.536450 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:55.536472 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:55.536537 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:55.537322 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:55.537325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:55.537533 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:10:55.537611 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:55.537618 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.538692 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.538696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:55.539282 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:55.539292 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.539887 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.539891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:55.539983 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:55.540459 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:10:55.540471 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:10:55.541799 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:10:55.544313 (+ 2514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:55.544317 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:55.544319 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:55.544654 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:55.545007 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:55.545010 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:55.545012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:55.545115 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:55.545872 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:10:55.545878 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:10:55.545988 (+ 110us) client_negotiation.cc:769] Sending connection context
0812 03:10:55.546089 (+ 101us) client_negotiation.cc:240] Negotiation successful
0812 03:10:55.546272 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 03:10:55.546751 10800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:55.536464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59722 (local address 127.26.250.131:36859)
0812 03:10:55.536853 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:55.536871 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:55.536891 (+ 20us) server_negotiation.cc:407] Connection header received
0812 03:10:55.536969 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:55.536976 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:55.537193 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:55.537346 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:55.537728 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.538567 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.539409 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.539765 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.539884 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:55.541979 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:55.541989 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:55.542001 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:55.542022 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:55.544199 (+ 2177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:55.544785 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:55.544793 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:55.544795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:55.544905 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:55.545235 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:55.545239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:55.545241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:55.545685 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:55.546086 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:55.546223 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 03:10:55.546471 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":167,"threads_started":1}
I20260812 03:10:55.548026 10726 tablet_service.cc:1514] Processing DeleteTablet for tablet ed78e734d8ae42d29604265b65243bee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:10:54 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:59722
I20260812 03:10:55.549239 10802 tablet_replica.cc:330] stopping tablet replica
I20260812 03:10:55.549582 10802 raft_consensus.cc:2229] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:10:55.549888 10802 raft_consensus.cc:2258] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:10:55.550246 10797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:55.545428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44986 (local address 127.26.250.190:33995)
0812 03:10:55.545550 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:55.545555 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:55.545851 (+ 296us) server_negotiation.cc:407] Connection header received
0812 03:10:55.545933 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:55.545936 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:55.546067 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:55.546171 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:10:55.546489 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.547243 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.548346 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.548713 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.548829 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:55.549338 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:10:55.549819 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:10:55.549966 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:55.550022 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 03:10:55.550095 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 03:10:55.551222 10802 ts_tablet_manager.cc:1905] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:10:55.550954 10305 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:44986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:10:55.554700 10305 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:10:55.555608 10802 ts_tablet_manager.cc:1918] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:10:55.555714 10802 log.cc:1196] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/ed78e734d8ae42d29604265b65243bee
I20260812 03:10:55.556073 10802 ts_tablet_manager.cc:1939] T ed78e734d8ae42d29604265b65243bee P 44f89d7ab5a8403cb0ca76caaafc37e4: Deleting consensus metadata
I20260812 03:10:55.558162 10293 catalog_manager.cc:4841] TS 44f89d7ab5a8403cb0ca76caaafc37e4 (127.26.250.131:36859): tablet ed78e734d8ae42d29604265b65243bee (table test-table [id=0a29426fc7794e5da958ec2f18a93cd6]) successfully deleted
I20260812 03:10:55.563199 10726 tablet_service.cc:1467] Processing CreateTablet for tablet 3a0bd1aec874402db2cb3dedf19a57bb (DEFAULT_TABLE table=test-table [id=569dff83f3a34c13a09618de33b38801]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:10:55.563432 10580 tablet_service.cc:1467] Processing CreateTablet for tablet 3a0bd1aec874402db2cb3dedf19a57bb (DEFAULT_TABLE table=test-table [id=569dff83f3a34c13a09618de33b38801]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:10:55.563889 10726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a0bd1aec874402db2cb3dedf19a57bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:10:55.563961 10580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a0bd1aec874402db2cb3dedf19a57bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:10:55.566919 10680 tablet_bootstrap.cc:492] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4: Bootstrap starting.
I20260812 03:10:55.566975 10535 tablet_bootstrap.cc:492] T 3a0bd1aec874402db2cb3dedf19a57bb P 119178da7c5a4e07abfbe0e3d159c425: Bootstrap starting.
I20260812 03:10:55.567565 10435 tablet_service.cc:1467] Processing CreateTablet for tablet 3a0bd1aec874402db2cb3dedf19a57bb (DEFAULT_TABLE table=test-table [id=569dff83f3a34c13a09618de33b38801]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:10:55.567996 10680 tablet_bootstrap.cc:654] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4: Neither blocks nor log segments found. Creating new log.
I20260812 03:10:55.568146 10535 tablet_bootstrap.cc:654] T 3a0bd1aec874402db2cb3dedf19a57bb P 119178da7c5a4e07abfbe0e3d159c425: Neither blocks nor log segments found. Creating new log.
I20260812 03:10:55.568210 10435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a0bd1aec874402db2cb3dedf19a57bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:10:55.569625 10680 tablet_bootstrap.cc:492] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4: No bootstrap required, opened a new log
I20260812 03:10:55.569763 10680 ts_tablet_manager.cc:1397] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:10:55.570230 10680 raft_consensus.cc:348] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:55.570406 10680 raft_consensus.cc:374] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:10:55.570487 10680 raft_consensus.cc:729] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44f89d7ab5a8403cb0ca76caaafc37e4, State: Initialized, Role: FOLLOWER
I20260812 03:10:55.570681 10680 consensus_queue.cc:260] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:55.571015 10680 ts_tablet_manager.cc:1428] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 03:10:55.571221 10389 tablet_bootstrap.cc:492] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59: Bootstrap starting.
I20260812 03:10:55.572337 10389 tablet_bootstrap.cc:654] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59: Neither blocks nor log segments found. Creating new log.
I20260812 03:10:55.572386 10535 tablet_bootstrap.cc:492] T 3a0bd1aec874402db2cb3dedf19a57bb P 119178da7c5a4e07abfbe0e3d159c425: No bootstrap required, opened a new log
I20260812 03:10:55.572504 10535 ts_tablet_manager.cc:1397] T 3a0bd1aec874402db2cb3dedf19a57bb P 119178da7c5a4e07abfbe0e3d159c425: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260812 03:10:55.573064 10535 raft_consensus.cc:348] T 3a0bd1aec874402db2cb3dedf19a57bb P 119178da7c5a4e07abfbe0e3d159c425 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:55.573204 10535 raft_consensus.cc:374] T 3a0bd1aec874402db2cb3dedf19a57bb P 119178da7c5a4e07abfbe0e3d159c425 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:10:55.573258 10535 raft_consensus.cc:729] T 3a0bd1aec874402db2cb3dedf19a57bb P 119178da7c5a4e07abfbe0e3d159c425 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 119178da7c5a4e07abfbe0e3d159c425, State: Initialized, Role: FOLLOWER
I20260812 03:10:55.573454 10535 consensus_queue.cc:260] T 3a0bd1aec874402db2cb3dedf19a57bb P 119178da7c5a4e07abfbe0e3d159c425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:55.573650 10389 tablet_bootstrap.cc:492] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59: No bootstrap required, opened a new log
I20260812 03:10:55.573782 10535 ts_tablet_manager.cc:1428] T 3a0bd1aec874402db2cb3dedf19a57bb P 119178da7c5a4e07abfbe0e3d159c425: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 03:10:55.573781 10389 ts_tablet_manager.cc:1397] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 03:10:55.574244 10389 raft_consensus.cc:348] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:55.574405 10389 raft_consensus.cc:374] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:10:55.574455 10389 raft_consensus.cc:729] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9269b47f18e146778c6d227fa1467f59, State: Initialized, Role: FOLLOWER
I20260812 03:10:55.574589 10389 consensus_queue.cc:260] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:55.574875 10389 ts_tablet_manager.cc:1428] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
W20260812 03:10:55.764285 10795 tablet.cc:2369] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:10:55.769914 10651 tablet.cc:2369] T 3a0bd1aec874402db2cb3dedf19a57bb P 119178da7c5a4e07abfbe0e3d159c425: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:10:55.798218 10504 tablet.cc:2369] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:10:55.829689 10799 raft_consensus.cc:482] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:10:55.829939 10799 raft_consensus.cc:504] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:55.831250 10799 leader_election.cc:288] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 119178da7c5a4e07abfbe0e3d159c425 (127.26.250.130:35585), 9269b47f18e146778c6d227fa1467f59 (127.26.250.129:41525)
I20260812 03:10:55.835909 10812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:55.832059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:53275 (local address 127.26.250.129:41525)
0812 03:10:55.832333 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:55.832338 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:55.832354 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:10:55.832476 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:55.832479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:55.832546 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:55.832697 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:10:55.832975 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.833508 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.834861 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.835319 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.835465 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:55.835626 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:55.835699 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 03:10:55.835772 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":98,"threads_started":1}
I20260812 03:10:55.835917 10810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:55.831927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:41525 (local address 127.26.250.131:53275)
0812 03:10:55.832229 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:55.832244 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:55.832345 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:55.832710 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:55.832713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:55.832731 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:10:55.832858 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:55.832863 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.833624 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.833629 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:55.834721 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:55.834732 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.835464 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.835468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:55.835555 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:55.835596 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 03:10:55.835674 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 03:10:55.835753 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":49,"thread_start_us":128,"threads_started":1}
I20260812 03:10:55.836094 10796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:55.831595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:35585 (local address 127.26.250.131:33407)
0812 03:10:55.831822 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:55.831842 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:55.831957 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:55.832509 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:55.832511 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:55.832527 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:10:55.832689 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:55.832695 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.833663 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.833667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:55.834778 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:55.834788 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.835569 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.835573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:55.835696 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:55.835720 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:10:55.835815 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 03:10:55.835892 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":44}
I20260812 03:10:55.836508 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:55.831720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:33407 (local address 127.26.250.130:35585)
0812 03:10:55.832124 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:55.832129 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:55.832149 (+ 20us) server_negotiation.cc:407] Connection header received
0812 03:10:55.832215 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:55.832221 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:55.832293 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:55.832487 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:10:55.832822 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.833543 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.834921 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.835428 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.835582 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:55.835749 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:55.835829 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 03:10:55.835910 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":170,"threads_started":1}
I20260812 03:10:55.836874 10456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a0bd1aec874402db2cb3dedf19a57bb" candidate_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9269b47f18e146778c6d227fa1467f59" is_pre_election: true
I20260812 03:10:55.837049 10592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a0bd1aec874402db2cb3dedf19a57bb" candidate_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "119178da7c5a4e07abfbe0e3d159c425" is_pre_election: true
I20260812 03:10:55.837292 10456 raft_consensus.cc:2454] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44f89d7ab5a8403cb0ca76caaafc37e4 in term 0.
I20260812 03:10:55.837347 10592 raft_consensus.cc:2454] T 3a0bd1aec874402db2cb3dedf19a57bb P 119178da7c5a4e07abfbe0e3d159c425 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44f89d7ab5a8403cb0ca76caaafc37e4 in term 0.
I20260812 03:10:55.837905 10676 leader_election.cc:302] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 119178da7c5a4e07abfbe0e3d159c425, 44f89d7ab5a8403cb0ca76caaafc37e4; no voters:
I20260812 03:10:55.838258 10799 raft_consensus.cc:2781] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:10:55.838415 10799 raft_consensus.cc:482] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:10:55.838485 10799 raft_consensus.cc:3037] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:10:55.839632 10799 raft_consensus.cc:504] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:55.840029 10799 leader_election.cc:288] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4 [CANDIDATE]: Term 1 election: Requested vote from peers 119178da7c5a4e07abfbe0e3d159c425 (127.26.250.130:35585), 9269b47f18e146778c6d227fa1467f59 (127.26.250.129:41525)
I20260812 03:10:55.840564 10592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a0bd1aec874402db2cb3dedf19a57bb" candidate_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "119178da7c5a4e07abfbe0e3d159c425"
I20260812 03:10:55.840641 10456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a0bd1aec874402db2cb3dedf19a57bb" candidate_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9269b47f18e146778c6d227fa1467f59"
I20260812 03:10:55.840718 10592 raft_consensus.cc:3037] T 3a0bd1aec874402db2cb3dedf19a57bb P 119178da7c5a4e07abfbe0e3d159c425 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:10:55.840826 10456 raft_consensus.cc:3037] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:10:55.841939 10592 raft_consensus.cc:2454] T 3a0bd1aec874402db2cb3dedf19a57bb P 119178da7c5a4e07abfbe0e3d159c425 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44f89d7ab5a8403cb0ca76caaafc37e4 in term 1.
I20260812 03:10:55.842165 10456 raft_consensus.cc:2454] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44f89d7ab5a8403cb0ca76caaafc37e4 in term 1.
I20260812 03:10:55.842264 10676 leader_election.cc:302] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 119178da7c5a4e07abfbe0e3d159c425, 44f89d7ab5a8403cb0ca76caaafc37e4; no voters:
I20260812 03:10:55.842515 10799 raft_consensus.cc:2781] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:10:55.842784 10799 raft_consensus.cc:686] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 1 LEADER]: Becoming Leader. State: Replica: 44f89d7ab5a8403cb0ca76caaafc37e4, State: Running, Role: LEADER
I20260812 03:10:55.843091 10799 consensus_queue.cc:237] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:55.846168 10306 catalog_manager.cc:5495] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 44f89d7ab5a8403cb0ca76caaafc37e4 (127.26.250.131). New cstate: current_term: 1 leader_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } health_report { overall_health: UNKNOWN } } }
I20260812 03:10:55.874521 10800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:55.870401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59736 (local address 127.26.250.131:36859)
0812 03:10:55.870551 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:55.870555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:55.870569 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:10:55.870699 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:55.870701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:55.870759 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:55.870853 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:10:55.871159 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.871811 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.872764 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:55.873114 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:55.873245 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:55.873605 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:10:55.874087 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:10:55.874285 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:55.874315 (+ 30us) server_negotiation.cc:299] Negotiation successful
0812 03:10:55.874396 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 03:10:55.879102 10592 raft_consensus.cc:1261] T 3a0bd1aec874402db2cb3dedf19a57bb P 119178da7c5a4e07abfbe0e3d159c425 [term 1 FOLLOWER]: Refusing update from remote peer 44f89d7ab5a8403cb0ca76caaafc37e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:10:55.879102 10456 raft_consensus.cc:1261] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Refusing update from remote peer 44f89d7ab5a8403cb0ca76caaafc37e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:10:55.879907 10813 consensus_queue.cc:1035] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:10:55.880100 10799 consensus_queue.cc:1035] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:10:55.886169 10815 mvcc.cc:205] Tried to move back new op lower bound from 7317521432073166848 to 7317521431935811584. Current Snapshot: MvccSnapshot[applied={T|T < 7317521432073166848}]
I20260812 03:10:55.888662 10817 mvcc.cc:205] Tried to move back new op lower bound from 7317521432073166848 to 7317521431935811584. Current Snapshot: MvccSnapshot[applied={T|T < 7317521432073166848 or (T in {7317521432073166848})}]
I20260812 03:10:55.896268 10305 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:44986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:10:55.896502 10305 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:44986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:10:55.899298 10305 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016: Sending DeleteTablet for 3 replicas of tablet 3a0bd1aec874402db2cb3dedf19a57bb
I20260812 03:10:55.900079 10726 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a0bd1aec874402db2cb3dedf19a57bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:10:55 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:59722
I20260812 03:10:55.900138 10580 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a0bd1aec874402db2cb3dedf19a57bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:10:55 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:41194
I20260812 03:10:55.900300 10802 tablet_replica.cc:330] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4: stopping tablet replica
I20260812 03:10:55.900352 10435 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a0bd1aec874402db2cb3dedf19a57bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:10:55 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:52524
I20260812 03:10:55.900442 10802 raft_consensus.cc:2229] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:10:55.900588 10824 tablet_replica.cc:330] T 3a0bd1aec874402db2cb3dedf19a57bb P 119178da7c5a4e07abfbe0e3d159c425: stopping tablet replica
I20260812 03:10:55.900746 10824 raft_consensus.cc:2229] T 3a0bd1aec874402db2cb3dedf19a57bb P 119178da7c5a4e07abfbe0e3d159c425 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:10:55.900830 10802 raft_consensus.cc:2258] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:10:55.900913 10824 raft_consensus.cc:2258] T 3a0bd1aec874402db2cb3dedf19a57bb P 119178da7c5a4e07abfbe0e3d159c425 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:10:55.900949 10825 tablet_replica.cc:330] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59: stopping tablet replica
I20260812 03:10:55.901093 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 10346
I20260812 03:10:55.901243 10825 raft_consensus.cc:2229] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:10:55.901435 10825 raft_consensus.cc:2258] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:10:55.903240 10824 ts_tablet_manager.cc:1905] T 3a0bd1aec874402db2cb3dedf19a57bb P 119178da7c5a4e07abfbe0e3d159c425: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:10:55.903272 10802 ts_tablet_manager.cc:1905] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:10:55.905432 10824 ts_tablet_manager.cc:1918] T 3a0bd1aec874402db2cb3dedf19a57bb P 119178da7c5a4e07abfbe0e3d159c425: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:10:55.905544 10824 log.cc:1196] T 3a0bd1aec874402db2cb3dedf19a57bb P 119178da7c5a4e07abfbe0e3d159c425: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/wal/wals/3a0bd1aec874402db2cb3dedf19a57bb
I20260812 03:10:55.905817 10824 ts_tablet_manager.cc:1939] T 3a0bd1aec874402db2cb3dedf19a57bb P 119178da7c5a4e07abfbe0e3d159c425: Deleting consensus metadata
I20260812 03:10:55.906563 10802 ts_tablet_manager.cc:1918] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:10:55.906668 10802 log.cc:1196] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/3a0bd1aec874402db2cb3dedf19a57bb
I20260812 03:10:55.906685 10292 catalog_manager.cc:4841] TS 119178da7c5a4e07abfbe0e3d159c425 (127.26.250.130:35585): tablet 3a0bd1aec874402db2cb3dedf19a57bb (table test-table [id=569dff83f3a34c13a09618de33b38801]) successfully deleted
I20260812 03:10:55.906963 10802 ts_tablet_manager.cc:1939] T 3a0bd1aec874402db2cb3dedf19a57bb P 44f89d7ab5a8403cb0ca76caaafc37e4: Deleting consensus metadata
I20260812 03:10:55.907876 10293 catalog_manager.cc:4841] TS 44f89d7ab5a8403cb0ca76caaafc37e4 (127.26.250.131:36859): tablet 3a0bd1aec874402db2cb3dedf19a57bb (table test-table [id=569dff83f3a34c13a09618de33b38801]) successfully deleted
W20260812 03:10:55.911010 10292 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.250.129:41525 (error 108)
I20260812 03:10:55.911716 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 10508
I20260812 03:10:55.912317 10510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:55.911938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:41525 (local address 127.0.0.1:52526)
0812 03:10:55.912141 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:55.912214 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.129:41525: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260812 03:10:55.913040 10292 catalog_manager.cc:4584] TS 9269b47f18e146778c6d227fa1467f59 (127.26.250.129:41525): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3a0bd1aec874402db2cb3dedf19a57bb: Network error: Client connection negotiation failed: client connection to 127.26.250.129:41525: connect: Connection refused (error 111)
I20260812 03:10:55.921077 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 10654
I20260812 03:10:55.930572 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 10273
I20260812 03:10:55.940026 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:33995
--webserver_interface=127.26.250.190
--webserver_port=33011
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:34825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:33995
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:10:56.065449 10827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:10:56.065783 10827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:10:56.065874 10827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:10:56.070493 10827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:10:56.070570 10827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:10:56.070590 10827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:10:56.070608 10827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:10:56.070626 10827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:10:56.076352 10827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:34825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:33995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:33995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=33011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:10:56.077602 10827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:10:56.078778 10827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:10:56.088508 10832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:56.088531 10835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:56.088737 10827 server_base.cc:1034] running on GCE node
W20260812 03:10:56.088508 10833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:56.089494 10827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:10:56.090780 10827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:10:56.091975 10827 hybrid_clock.cc:648] HybridClock initialized: now 1786504256091943 us; error 47 us; skew 500 ppm
Aug 12 03:10:56 dist-test-slave-zpfg krb5kdc[9704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504256, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:10:56.095153 10827 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:10:56.096175 10827 webserver.cc:460] Webserver started at http://127.26.250.190:33011/ using document root <none> and password file <none>
I20260812 03:10:56.097033 10827 fs_manager.cc:359] Metadata directory not provided
I20260812 03:10:56.097174 10827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:10:56.101810 10827 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:10:56.104666 10843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:56.106546 10827 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:10:56.106712 10827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "39dbec4451b84572a24d4826c5120016"
format_stamp: "Formatted at 2026-08-12 03:10:53 on dist-test-slave-zpfg"
server_key: "30e312031a8c0ad93255f94ca3f14b7d"
server_key_iv: "9466b21980e80429142fb7f0ddbac666"
server_key_version: "encryptionkey@0"
I20260812 03:10:56.107363 10827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:10:56.129981 10827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:10:56.133987 10827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:10:56.134258 10827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:10:56.142931 10827 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:33995
I20260812 03:10:56.142948 10897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:33995 every 8 connection(s)
I20260812 03:10:56.144191 10827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:10:56.148742 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 10827
I20260812 03:10:56.149339 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:41525
--local_ip_for_outbound_sockets=127.26.250.129
--tserver_master_addrs=127.26.250.190:33995
--webserver_port=46335
--webserver_interface=127.26.250.129
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:34825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:10:56.149398 10898 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:10:56.151453 10898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016: Bootstrap starting.
I20260812 03:10:56.180687 10898 log.cc:824] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016: Log is configured to *not* fsync() on all Append() calls
I20260812 03:10:56.196957 10898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:10:56.197491 10898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016: Bootstrap complete.
I20260812 03:10:56.204398 10898 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39dbec4451b84572a24d4826c5120016" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 33995 } }
I20260812 03:10:56.205029 10898 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39dbec4451b84572a24d4826c5120016, State: Initialized, Role: FOLLOWER
I20260812 03:10:56.205685 10898 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39dbec4451b84572a24d4826c5120016" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 33995 } }
I20260812 03:10:56.205861 10898 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:10:56.205938 10898 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:10:56.206048 10898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [term 2 FOLLOWER]: Advancing to term 3
I20260812 03:10:56.209231 10898 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39dbec4451b84572a24d4826c5120016" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 33995 } }
I20260812 03:10:56.209663 10898 leader_election.cc:302] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39dbec4451b84572a24d4826c5120016; no voters:
I20260812 03:10:56.210073 10898 leader_election.cc:288] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 03:10:56.210250 10902 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 03:10:56.210649 10902 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [term 3 LEADER]: Becoming Leader. State: Replica: 39dbec4451b84572a24d4826c5120016, State: Running, Role: LEADER
I20260812 03:10:56.210989 10902 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39dbec4451b84572a24d4826c5120016" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 33995 } }
I20260812 03:10:56.211555 10898 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:10:56.212746 10904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39dbec4451b84572a24d4826c5120016. Latest consensus state: current_term: 3 leader_uuid: "39dbec4451b84572a24d4826c5120016" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39dbec4451b84572a24d4826c5120016" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 33995 } } }
I20260812 03:10:56.212774 10903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "39dbec4451b84572a24d4826c5120016" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39dbec4451b84572a24d4826c5120016" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 33995 } } }
I20260812 03:10:56.212903 10903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [sys.catalog]: This master's current role is: LEADER
I20260812 03:10:56.212903 10904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016 [sys.catalog]: This master's current role is: LEADER
I20260812 03:10:56.217525 10917 catalog_manager.cc:1259] Loaded cluster ID: 77b1b5ae15974668a2ad4b660461cc9a
I20260812 03:10:56.217612 10917 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016: loading cluster ID for follower catalog manager: success
W20260812 03:10:56.218924 10917 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 39dbec4451b84572a24d4826c5120016: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260812 03:10:56.219023 10917 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260812 03:10:56.219250 10911 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:10:56.223438 10911 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:10:56.223990 10911 catalog_manager.cc:1259] Loaded cluster ID: 77b1b5ae15974668a2ad4b660461cc9a
I20260812 03:10:56.224056 10911 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 03:10:56.224701 10911 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ff02d5459ad google::LogMessage::Fail() at ??:0
@ 0x7ff02d54888a google::LogMessage::SendToLog() at ??:0
@ 0x7ff02d5454de google::LogMessage::Flush() at ??:0
@ 0x7ff02d546a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ff037e84fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7ff037e85d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ff037e82579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7ff037ec6640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7ff02f7973d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ff02f797c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7ff02f7993f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7ff02f788512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7ff030dcc6db start_thread at ??:0
@ 0x7ff02c04361f clone at ??:0
W20260812 03:10:56.327276 10900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:10:56.327584 10900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:10:56.327669 10900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:10:56.332067 10900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:10:56.332139 10900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:10:56.332245 10900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:10:56.338088 10900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:34825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:41525
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=46335
--tserver_master_addrs=127.26.250.190:33995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:10:56.339334 10900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:10:56.340437 10900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:10:56.352005 10939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:56.352005 10940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:56.352349 10900 server_base.cc:1034] running on GCE node
W20260812 03:10:56.352069 10942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:56.353008 10900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:10:56.353906 10900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:10:56.355124 10900 hybrid_clock.cc:648] HybridClock initialized: now 1786504256355102 us; error 50 us; skew 500 ppm
Aug 12 03:10:56 dist-test-slave-zpfg krb5kdc[9704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504256, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:10:56.358520 10900 init.cc:377] Logged in from keytab as kudu/127.26.250.129@KRBTEST.COM (short username kudu)
I20260812 03:10:56.359637 10900 webserver.cc:460] Webserver started at http://127.26.250.129:46335/ using document root <none> and password file <none>
I20260812 03:10:56.360241 10900 fs_manager.cc:359] Metadata directory not provided
I20260812 03:10:56.360308 10900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:10:56.364852 10900 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:10:56.367486 10949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:56.369146 10900 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:10:56.369406 10900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "9269b47f18e146778c6d227fa1467f59"
format_stamp: "Formatted at 2026-08-12 03:10:53 on dist-test-slave-zpfg"
server_key: "0c51f5dae1475c5b8c70802a1f223a08"
server_key_iv: "e9fe2200106f8117943c4dd77328fecb"
server_key_version: "encryptionkey@0"
I20260812 03:10:56.370031 10900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:10:56.434711 10900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:10:56.438549 10900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:10:56.438776 10900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:10:56.439437 10900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:10:56.440605 10957 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 03:10:56.445123 10900 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 03:10:56.445251 10900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260812 03:10:56.445363 10900 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 03:10:56.447412 10900 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 03:10:56.447497 10900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260812 03:10:56.447830 10957 tablet_bootstrap.cc:492] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59: Bootstrap starting.
I20260812 03:10:56.459048 10900 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:41525
I20260812 03:10:56.459156 11071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:41525 every 8 connection(s)
I20260812 03:10:56.460189 10900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:10:56.464583 11074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:56.462702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.129:47031)
0812 03:10:56.463288 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:56.463462 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":132,"threads_started":1}
W20260812 03:10:56.465283 11072 heartbeater.cc:644] Failed to heartbeat to 127.26.250.190:33995 (0 consecutive failures): Network error: Failed to ping master at 127.26.250.190:33995: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
I20260812 03:10:56.466115 11074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:56.465790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.129:54697)
0812 03:10:56.465969 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:56.466041 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 03:10:56.466807 11074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:56.466567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.129:47627)
0812 03:10:56.466691 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:56.466743 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 03:10:56.467024 11072 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 03:10:56.469372 10957 log.cc:824] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59: Log is configured to *not* fsync() on all Append() calls
I20260812 03:10:56.470534 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 10900
I20260812 03:10:56.471050 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:35585
--local_ip_for_outbound_sockets=127.26.250.130
--tserver_master_addrs=127.26.250.190:33995
--webserver_port=42481
--webserver_interface=127.26.250.130
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:34825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:10:56.477924 10957 tablet_bootstrap.cc:492] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:10:56.478386 10957 tablet_bootstrap.cc:492] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59: Bootstrap complete.
I20260812 03:10:56.479681 10957 ts_tablet_manager.cc:1397] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59: Time spent bootstrapping tablet: real 0.032s user 0.021s sys 0.009s
I20260812 03:10:56.482724 10957 raft_consensus.cc:348] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:56.483242 10957 raft_consensus.cc:729] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9269b47f18e146778c6d227fa1467f59, State: Initialized, Role: FOLLOWER
I20260812 03:10:56.483752 10957 consensus_queue.cc:260] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:56.484890 10957 ts_tablet_manager.cc:1428] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 03:10:56.485168 11074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:56.484841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.129:37983)
0812 03:10:56.484996 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:56.485074 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 03:10:56.485882 11074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:56.485634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.129:38545)
0812 03:10:56.485756 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:56.485815 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 03:10:56.597469 11076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:10:56.597769 11076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:10:56.597816 11076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:10:56.602077 11076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:10:56.602157 11076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:10:56.602257 11076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:10:56.607856 11076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:34825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:35585
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=42481
--tserver_master_addrs=127.26.250.190:33995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:10:56.609378 11076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:10:56.610497 11076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:10:56.621305 11082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:56.621342 11083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:56.621330 11085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:56.621570 11076 server_base.cc:1034] running on GCE node
I20260812 03:10:56.622030 11076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:10:56.622799 11076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:10:56.623982 11076 hybrid_clock.cc:648] HybridClock initialized: now 1786504256623965 us; error 30 us; skew 500 ppm
Aug 12 03:10:56 dist-test-slave-zpfg krb5kdc[9704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504256, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:10:56.627066 11076 init.cc:377] Logged in from keytab as kudu/127.26.250.130@KRBTEST.COM (short username kudu)
I20260812 03:10:56.628096 11076 webserver.cc:460] Webserver started at http://127.26.250.130:42481/ using document root <none> and password file <none>
I20260812 03:10:56.628748 11076 fs_manager.cc:359] Metadata directory not provided
I20260812 03:10:56.628865 11076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:10:56.633324 11076 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:10:56.635769 11092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:56.637436 11076 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:10:56.637607 11076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "119178da7c5a4e07abfbe0e3d159c425"
format_stamp: "Formatted at 2026-08-12 03:10:53 on dist-test-slave-zpfg"
server_key: "e723187ff2b0406dd548ae08285ca77d"
server_key_iv: "c8c821b37645176f9ddca47f84de2a15"
server_key_version: "encryptionkey@0"
I20260812 03:10:56.638284 11076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:10:56.649209 11076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:10:56.653263 11076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:10:56.653519 11076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:10:56.654253 11076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:10:56.655356 11076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:10:56.655441 11076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:56.655521 11076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:10:56.655578 11076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:56.666553 11076 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:35585
I20260812 03:10:56.666594 11212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:35585 every 8 connection(s)
I20260812 03:10:56.667728 11076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:10:56.669915 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 11076
I20260812 03:10:56.670367 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:36859
--local_ip_for_outbound_sockets=127.26.250.131
--tserver_master_addrs=127.26.250.190:33995
--webserver_port=35469
--webserver_interface=127.26.250.131
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:34825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:10:56.672206 11215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:56.670158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.130:39939)
0812 03:10:56.670838 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:56.671453 (+ 615us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":213,"threads_started":1}
W20260812 03:10:56.672761 11213 heartbeater.cc:644] Failed to heartbeat to 127.26.250.190:33995 (0 consecutive failures): Network error: Failed to ping master at 127.26.250.190:33995: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
I20260812 03:10:56.673591 11215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:56.673254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.130:56455)
0812 03:10:56.673441 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:56.673514 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260812 03:10:56.677006 11215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:56.674962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.130:40631)
0812 03:10:56.676854 (+ 1892us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:56.676923 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1784}
W20260812 03:10:56.677328 11213 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:10:56.802016 11216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:10:56.802598 11216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:10:56.802711 11216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:10:56.807597 11216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:10:56.807705 11216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:10:56.807816 11216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:10:56.814231 11216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:34825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:36859
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=35469
--tserver_master_addrs=127.26.250.190:33995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:10:56.815618 11216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:10:56.816947 11216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:10:56.828606 11224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:56.828606 11226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:10:56.828657 11221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:10:56.829149 11216 server_base.cc:1034] running on GCE node
I20260812 03:10:56.829690 11216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:10:56.830507 11216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:10:56.831712 11216 hybrid_clock.cc:648] HybridClock initialized: now 1786504256831658 us; error 63 us; skew 500 ppm
Aug 12 03:10:56 dist-test-slave-zpfg krb5kdc[9704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504256, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:10:56.834959 11216 init.cc:377] Logged in from keytab as kudu/127.26.250.131@KRBTEST.COM (short username kudu)
I20260812 03:10:56.835881 11216 webserver.cc:460] Webserver started at http://127.26.250.131:35469/ using document root <none> and password file <none>
I20260812 03:10:56.836494 11216 fs_manager.cc:359] Metadata directory not provided
I20260812 03:10:56.836565 11216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:10:56.840704 11216 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:10:56.843015 11234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:56.844461 11216 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:10:56.844619 11216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "44f89d7ab5a8403cb0ca76caaafc37e4"
format_stamp: "Formatted at 2026-08-12 03:10:54 on dist-test-slave-zpfg"
server_key: "dad9c07676f75f06f34b2c7c11ce4d54"
server_key_iv: "22a77a5e0a9aa97ad3ec7abb825af631"
server_key_version: "encryptionkey@0"
I20260812 03:10:56.845198 11216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:10:56.866303 11216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:10:56.870052 11216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:10:56.870285 11216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:10:56.871117 11216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:10:56.871984 11216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:10:56.872033 11216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:56.872076 11216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:10:56.872092 11216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:10:56.882441 11216 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:36859
I20260812 03:10:56.882480 11352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:36859 every 8 connection(s)
I20260812 03:10:56.883607 11216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:10:56.888144 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:56.886103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.131:50351)
0812 03:10:56.886776 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:56.887245 (+ 469us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":183,"threads_started":1}
W20260812 03:10:56.888720 11353 heartbeater.cc:644] Failed to heartbeat to 127.26.250.190:33995 (0 consecutive failures): Network error: Failed to ping master at 127.26.250.190:33995: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
I20260812 03:10:56.889580 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:56.889222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.131:45709)
0812 03:10:56.889454 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:56.889515 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
I20260812 03:10:56.890511 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 11216
I20260812 03:10:56.890555 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:56.890125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.131:48463)
0812 03:10:56.890419 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:56.890480 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183}
W20260812 03:10:56.890851 11353 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 03:10:57.487272 11358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:57.486746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.129:48373)
0812 03:10:57.487064 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:57.487162 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":93,"threads_started":1}
I20260812 03:10:57.647552 11359 raft_consensus.cc:482] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:10:57.647770 11359 raft_consensus.cc:504] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:57.648957 11359 leader_election.cc:288] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 44f89d7ab5a8403cb0ca76caaafc37e4 (127.26.250.131:36859), 119178da7c5a4e07abfbe0e3d159c425 (127.26.250.130:35585)
Aug 12 03:10:57 dist-test-slave-zpfg krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504256, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for kudu/127.26.250.130@KRBTEST.COM
Aug 12 03:10:57 dist-test-slave-zpfg krb5kdc[9704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504256, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for kudu/127.26.250.131@KRBTEST.COM
I20260812 03:10:57.664428 11362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:57.649501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:53195 (local address 127.26.250.131:36859)
0812 03:10:57.650241 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:57.650249 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:57.650282 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:10:57.652963 (+ 2681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:57.652970 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:57.653292 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:57.653599 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:57.654036 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:57.655068 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:57.655891 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:57.656381 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:57.656559 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:57.659427 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:57.659433 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:57.659448 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:57.659485 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:57.661864 (+ 2379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:57.662319 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:57.662325 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:57.662331 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:57.662479 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:57.662805 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:57.662810 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:57.662811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:57.663238 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:57.663390 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:57.663956 (+ 566us) server_negotiation.cc:299] Negotiation successful
0812 03:10:57.664225 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":60,"threads_started":1}
I20260812 03:10:57.664415 11360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:57.649264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:36859 (local address 127.26.250.129:53195)
0812 03:10:57.649816 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:57.649849 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:57.650371 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:57.653527 (+ 3156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:57.653530 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:57.653746 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:10:57.653859 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:57.653867 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:57.655206 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:57.655210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:57.655758 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:57.655766 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:57.656516 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:57.656522 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:57.656616 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:57.657299 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:10:57.657313 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:10:57.659210 (+ 1897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:10:57.661988 (+ 2778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:57.661993 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:57.661996 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:57.662196 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:57.662587 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:57.662590 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:57.662592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:57.662707 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:57.663371 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:10:57.663377 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:10:57.663642 (+ 265us) client_negotiation.cc:769] Sending connection context
0812 03:10:57.663835 (+ 193us) client_negotiation.cc:240] Negotiation successful
0812 03:10:57.664237 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":59,"spinlock_wait_cycles":9344}
I20260812 03:10:57.664897 11358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:57.649277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:35585 (local address 127.26.250.129:35263)
0812 03:10:57.649819 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:57.649849 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 03:10:57.650368 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:10:57.652069 (+ 1701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:10:57.652077 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:10:57.652485 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:10:57.653061 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:57.653079 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:57.654574 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:57.654577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:57.655414 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:10:57.655424 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:10:57.656228 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:10:57.656232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:10:57.656442 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:57.657113 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:10:57.657140 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:10:57.658979 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:10:57.661794 (+ 2815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:57.661804 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:57.661819 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:57.662081 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:57.662530 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:10:57.662534 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:10:57.662536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:10:57.662680 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:10:57.663371 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:10:57.663377 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:10:57.663642 (+ 265us) client_negotiation.cc:769] Sending connection context
0812 03:10:57.663836 (+ 194us) client_negotiation.cc:240] Negotiation successful
0812 03:10:57.664332 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"mutex_wait_us":279,"spinlock_wait_cycles":62336,"thread_start_us":81,"threads_started":1}
I20260812 03:10:57.664456 11361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:57.649467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:35263 (local address 127.26.250.130:35585)
0812 03:10:57.649952 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0812 03:10:57.649966 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:10:57.650013 (+ 47us) server_negotiation.cc:407] Connection header received
0812 03:10:57.651298 (+ 1285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:10:57.651305 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:10:57.651684 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:10:57.651952 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:10:57.653302 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:57.654425 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:57.655556 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:10:57.656068 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:10:57.656278 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:10:57.659281 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:10:57.659291 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:10:57.659310 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:10:57.659415 (+ 105us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:10:57.661640 (+ 2225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:57.662223 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:57.662228 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:57.662233 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:57.662430 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:10:57.662781 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:10:57.662786 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:10:57.662791 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:10:57.663238 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:10:57.663394 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 03:10:57.663958 (+ 564us) server_negotiation.cc:299] Negotiation successful
0812 03:10:57.664226 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":90,"threads_started":1}
W20260812 03:10:57.667193 10953 leader_election.cc:341] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 44f89d7ab5a8403cb0ca76caaafc37e4 (127.26.250.131:36859): Not found: Tablet not found: 3a0bd1aec874402db2cb3dedf19a57bb
W20260812 03:10:57.667891 10952 leader_election.cc:341] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 119178da7c5a4e07abfbe0e3d159c425 (127.26.250.130:35585): Not found: Tablet not found: 3a0bd1aec874402db2cb3dedf19a57bb
I20260812 03:10:57.667990 10952 leader_election.cc:302] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9269b47f18e146778c6d227fa1467f59; no voters: 119178da7c5a4e07abfbe0e3d159c425, 44f89d7ab5a8403cb0ca76caaafc37e4
I20260812 03:10:57.668301 11359 raft_consensus.cc:2726] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:10:57.678480 11363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:57.677999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.130:33337)
0812 03:10:57.678244 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:57.678299 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":70,"threads_started":1}
I20260812 03:10:57.892064 11364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:57.891551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.131:57129)
0812 03:10:57.891885 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:57.891952 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":106,"threads_started":1}
I20260812 03:10:58.488617 11365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:58.488087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.129:55709)
0812 03:10:58.488442 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:58.488512 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":85,"threads_started":1}
I20260812 03:10:58.679773 11366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:58.679336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.130:36161)
0812 03:10:58.679605 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:58.679667 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":81,"threads_started":1}
I20260812 03:10:58.893530 11367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:58.892990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.131:55213)
0812 03:10:58.893333 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:58.893399 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":123,"threads_started":1}
I20260812 03:10:59.490105 11368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:59.489579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.129:58053)
0812 03:10:59.489942 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:59.490006 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":100,"threads_started":1}
I20260812 03:10:59.600301 11369 raft_consensus.cc:482] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:10:59.600476 11369 raft_consensus.cc:504] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:10:59.601001 11369 leader_election.cc:288] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 44f89d7ab5a8403cb0ca76caaafc37e4 (127.26.250.131:36859), 119178da7c5a4e07abfbe0e3d159c425 (127.26.250.130:35585)
W20260812 03:10:59.601969 10952 leader_election.cc:341] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 119178da7c5a4e07abfbe0e3d159c425 (127.26.250.130:35585): Not found: Tablet not found: 3a0bd1aec874402db2cb3dedf19a57bb
W20260812 03:10:59.602097 10953 leader_election.cc:341] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 44f89d7ab5a8403cb0ca76caaafc37e4 (127.26.250.131:36859): Not found: Tablet not found: 3a0bd1aec874402db2cb3dedf19a57bb
I20260812 03:10:59.602233 10953 leader_election.cc:302] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9269b47f18e146778c6d227fa1467f59; no voters: 119178da7c5a4e07abfbe0e3d159c425, 44f89d7ab5a8403cb0ca76caaafc37e4
I20260812 03:10:59.602451 11369 raft_consensus.cc:2726] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:10:59.681447 11370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:59.680835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.130:40107)
0812 03:10:59.681178 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:59.681252 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":112,"threads_started":1}
I20260812 03:10:59.895102 11371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:10:59.894514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.131:40539)
0812 03:10:59.894888 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0812 03:10:59.894979 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":81,"threads_started":1}
I20260812 03:11:00.492008 11372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:00.491073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.129:40457)
0812 03:11:00.491825 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:00.491886 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":99,"threads_started":1}
I20260812 03:11:00.683084 11373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:00.682537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.130:52573)
0812 03:11:00.682918 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:00.682982 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":104,"threads_started":1}
I20260812 03:11:00.896581 11374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:00.896072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.131:58603)
0812 03:11:00.896422 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:00.896481 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":98,"threads_started":1}
I20260812 03:11:01.493671 11375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:01.493114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.129:38317)
0812 03:11:01.493498 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:01.493568 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":103,"threads_started":1}
I20260812 03:11:01.606882 11376 raft_consensus.cc:482] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:11:01.607074 11376 raft_consensus.cc:504] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:11:01.607594 11376 leader_election.cc:288] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 44f89d7ab5a8403cb0ca76caaafc37e4 (127.26.250.131:36859), 119178da7c5a4e07abfbe0e3d159c425 (127.26.250.130:35585)
W20260812 03:11:01.608467 10953 leader_election.cc:341] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 44f89d7ab5a8403cb0ca76caaafc37e4 (127.26.250.131:36859): Not found: Tablet not found: 3a0bd1aec874402db2cb3dedf19a57bb
W20260812 03:11:01.608568 10952 leader_election.cc:341] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 119178da7c5a4e07abfbe0e3d159c425 (127.26.250.130:35585): Not found: Tablet not found: 3a0bd1aec874402db2cb3dedf19a57bb
I20260812 03:11:01.608691 10952 leader_election.cc:302] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9269b47f18e146778c6d227fa1467f59; no voters: 119178da7c5a4e07abfbe0e3d159c425, 44f89d7ab5a8403cb0ca76caaafc37e4
I20260812 03:11:01.608906 11376 raft_consensus.cc:2726] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:11:01.684840 11377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:01.684156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.130:45877)
0812 03:11:01.684588 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:01.684674 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":131,"threads_started":1}
I20260812 03:11:01.898031 11378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:01.897561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.131:35515)
0812 03:11:01.897862 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:01.897932 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":87,"threads_started":1}
I20260812 03:11:02.495260 11379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:02.494761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.129:58807)
0812 03:11:02.495063 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:02.495134 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":82,"threads_started":1}
I20260812 03:11:02.689054 11380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:02.688540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.130:56739)
0812 03:11:02.688872 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:02.688942 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":134,"threads_started":1}
I20260812 03:11:02.899559 11381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:02.898984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.131:37007)
0812 03:11:02.899381 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:02.899455 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":89,"threads_started":1}
I20260812 03:11:03.496575 11382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:03.496083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.129:48231)
0812 03:11:03.496399 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:03.496465 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":96,"threads_started":1}
I20260812 03:11:03.690614 11383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:03.690007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.130:41915)
0812 03:11:03.690445 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:03.690509 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":161,"threads_started":1}
I20260812 03:11:03.900952 11384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:03.900418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.131:46909)
0812 03:11:03.900784 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:03.900844 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":86,"threads_started":1}
I20260812 03:11:04.157819 11385 raft_consensus.cc:482] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:11:04.157995 11385 raft_consensus.cc:504] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:11:04.158564 11385 leader_election.cc:288] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 44f89d7ab5a8403cb0ca76caaafc37e4 (127.26.250.131:36859), 119178da7c5a4e07abfbe0e3d159c425 (127.26.250.130:35585)
W20260812 03:11:04.159337 10953 leader_election.cc:341] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 44f89d7ab5a8403cb0ca76caaafc37e4 (127.26.250.131:36859): Not found: Tablet not found: 3a0bd1aec874402db2cb3dedf19a57bb
W20260812 03:11:04.159466 10952 leader_election.cc:341] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 119178da7c5a4e07abfbe0e3d159c425 (127.26.250.130:35585): Not found: Tablet not found: 3a0bd1aec874402db2cb3dedf19a57bb
I20260812 03:11:04.159612 10952 leader_election.cc:302] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9269b47f18e146778c6d227fa1467f59; no voters: 119178da7c5a4e07abfbe0e3d159c425, 44f89d7ab5a8403cb0ca76caaafc37e4
I20260812 03:11:04.159796 11385 raft_consensus.cc:2726] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:11:04.498018 11386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:04.497545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.129:59741)
0812 03:11:04.497850 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:04.497915 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":95,"threads_started":1}
I20260812 03:11:04.691984 11387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:04.691503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.130:40633)
0812 03:11:04.691812 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:04.691880 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":82,"threads_started":1}
I20260812 03:11:04.902746 11388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:04.902156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.131:57707)
0812 03:11:04.902545 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:04.902630 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":111,"threads_started":1}
I20260812 03:11:05.499541 11389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:05.499069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.129:52799)
0812 03:11:05.499368 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:05.499438 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":83,"threads_started":1}
I20260812 03:11:05.698789 11390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:05.692909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.130:46745)
0812 03:11:05.698592 (+ 5683us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:05.698676 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5582,"thread_start_us":89,"threads_started":1}
I20260812 03:11:05.904309 11391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:05.903671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.131:52611)
0812 03:11:05.904047 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:05.904107 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":101,"threads_started":1}
I20260812 03:11:06.501104 11392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:06.500546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.129:48889)
0812 03:11:06.500930 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:06.500999 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":116,"threads_started":1}
I20260812 03:11:06.700423 11393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:06.699881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.130:46505)
0812 03:11:06.700203 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:06.700284 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":87,"threads_started":1}
I20260812 03:11:06.905982 11394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:06.905440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.131:38781)
0812 03:11:06.905788 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:06.905872 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":86,"threads_started":1}
I20260812 03:11:07.502738 11395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:07.502102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.129:43647)
0812 03:11:07.502569 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:07.502634 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":152,"threads_started":1}
I20260812 03:11:07.701934 11396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:07.701384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.130:60953)
0812 03:11:07.701715 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:07.701826 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":98,"threads_started":1}
I20260812 03:11:07.907618 11397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:07.907040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.131:37997)
0812 03:11:07.907444 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:07.907514 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":106,"threads_started":1}
I20260812 03:11:08.504076 11398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:08.503571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.129:35881)
0812 03:11:08.503897 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:08.503962 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":99,"threads_started":1}
I20260812 03:11:08.703414 11399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:08.702960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.130:50791)
0812 03:11:08.703247 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:08.703313 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":85,"threads_started":1}
I20260812 03:11:08.909260 11400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:08.908656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.131:34427)
0812 03:11:08.909076 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:08.909150 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":186,"threads_started":1}
I20260812 03:11:09.505841 11401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:09.505209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.129:58839)
0812 03:11:09.505605 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:09.505714 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":131,"threads_started":1}
I20260812 03:11:09.704953 11402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:09.704425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.130:33491)
0812 03:11:09.704757 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:09.704831 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":87,"threads_started":1}
I20260812 03:11:09.910881 11403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:09.910330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.131:37151)
0812 03:11:09.910704 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:09.910780 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":88,"threads_started":1}
I20260812 03:11:10.208942 11405 raft_consensus.cc:482] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:11:10.209096 11405 raft_consensus.cc:504] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44f89d7ab5a8403cb0ca76caaafc37e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 36859 } } peers { permanent_uuid: "119178da7c5a4e07abfbe0e3d159c425" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 35585 } } peers { permanent_uuid: "9269b47f18e146778c6d227fa1467f59" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 41525 } }
I20260812 03:11:10.209578 11405 leader_election.cc:288] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 44f89d7ab5a8403cb0ca76caaafc37e4 (127.26.250.131:36859), 119178da7c5a4e07abfbe0e3d159c425 (127.26.250.130:35585)
W20260812 03:11:10.210382 10953 leader_election.cc:341] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 44f89d7ab5a8403cb0ca76caaafc37e4 (127.26.250.131:36859): Not found: Tablet not found: 3a0bd1aec874402db2cb3dedf19a57bb
W20260812 03:11:10.210537 10952 leader_election.cc:341] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 119178da7c5a4e07abfbe0e3d159c425 (127.26.250.130:35585): Not found: Tablet not found: 3a0bd1aec874402db2cb3dedf19a57bb
I20260812 03:11:10.210630 10952 leader_election.cc:302] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9269b47f18e146778c6d227fa1467f59; no voters: 119178da7c5a4e07abfbe0e3d159c425, 44f89d7ab5a8403cb0ca76caaafc37e4
I20260812 03:11:10.210794 11405 raft_consensus.cc:2726] T 3a0bd1aec874402db2cb3dedf19a57bb P 9269b47f18e146778c6d227fa1467f59 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:11:10.507709 11406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:10.507108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.129:55961)
0812 03:11:10.507522 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:10.507591 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":135,"threads_started":1}
I20260812 03:11:10.706787 11407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:10.706035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.130:40273)
0812 03:11:10.706565 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:10.706639 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":99,"threads_started":1}
I20260812 03:11:10.912581 11408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:10.911908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.131:38821)
0812 03:11:10.912359 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:10.912448 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":239,"threads_started":1}
I20260812 03:11:11.509321 11409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:11.508832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.129:34259)
0812 03:11:11.509138 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:11.509208 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":82,"threads_started":1}
I20260812 03:11:11.708837 11410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:11.708184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:33995 (local address 127.26.250.130:59401)
0812 03:11:11.708563 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:11.708670 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:33995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":95,"threads_started":1}
I20260812 03:11:11.892094 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 10900
I20260812 03:11:11.902381 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 11076
I20260812 03:11:11.911397 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 11216
2026-08-12T03:11:11Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (20050 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:11:11 dist-test-slave-zpfg krb5kdc[11413](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:11:11 dist-test-slave-zpfg krb5kdc[11413](info): set up 2 sockets
Aug 12 03:11:11 dist-test-slave-zpfg krb5kdc[11413](info): commencing operation
krb5kdc: starting...
W20260812 03:11:13.012849 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:11:13 dist-test-slave-zpfg krb5kdc[11413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:11:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:11:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:11:13.162530 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:44865
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:40161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:44865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:11:13.283677 11426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:11:13.284037 11426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:11:13.284116 11426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:11:13.288365 11426 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 03:11:13.288465 11426 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 03:11:13.288489 11426 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 03:11:13.288514 11426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:11:13.288555 11426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:11:13.288623 11426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:11:13.288676 11426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:11:13.288712 11426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:11:13.294412 11426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:40161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:44865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:44865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:11:13.295697 11426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:11:13.296841 11426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:11:13.307132 11434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:11:13.307197 11426 server_base.cc:1034] running on GCE node
W20260812 03:11:13.307206 11432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:11:13.307466 11431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:11:13.308301 11426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:11:13.309640 11426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:11:13.310891 11426 hybrid_clock.cc:648] HybridClock initialized: now 1786504273310847 us; error 63 us; skew 500 ppm
Aug 12 03:11:13 dist-test-slave-zpfg krb5kdc[11413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504273, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:11:13.314498 11426 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:11:13.315613 11426 webserver.cc:460] Webserver started at http://127.26.250.190:39479/ using document root <none> and password file <none>
I20260812 03:11:13.316252 11426 fs_manager.cc:359] Metadata directory not provided
I20260812 03:11:13.316324 11426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:11:13.316514 11426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:11:13.319036 11426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "8f7ac59b76224979a06891a373d00f2a"
format_stamp: "Formatted at 2026-08-12 03:11:13 on dist-test-slave-zpfg"
server_key: "4c1016cd34d3c107df325752d87c2211"
server_key_iv: "2e8a470e634bbb3118c8cc1e88db6735"
server_key_version: "encryptionkey@0"
I20260812 03:11:13.319674 11426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "8f7ac59b76224979a06891a373d00f2a"
format_stamp: "Formatted at 2026-08-12 03:11:13 on dist-test-slave-zpfg"
server_key: "4c1016cd34d3c107df325752d87c2211"
server_key_iv: "2e8a470e634bbb3118c8cc1e88db6735"
server_key_version: "encryptionkey@0"
I20260812 03:11:13.324084 11426 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:11:13.327085 11441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:11:13.328804 11426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:11:13.328945 11426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "8f7ac59b76224979a06891a373d00f2a"
format_stamp: "Formatted at 2026-08-12 03:11:13 on dist-test-slave-zpfg"
server_key: "4c1016cd34d3c107df325752d87c2211"
server_key_iv: "2e8a470e634bbb3118c8cc1e88db6735"
server_key_version: "encryptionkey@0"
I20260812 03:11:13.329059 11426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:11:13.353605 11426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:11:13.357234 11426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:11:13.357432 11426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:11:13.365141 11426 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:44865
I20260812 03:11:13.365541 11496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:44865 every 8 connection(s)
I20260812 03:11:13.366228 11426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:11:13.369400 11497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:11:13.371605 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 11426
I20260812 03:11:13.371719 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:11:13.372049 27626 external_mini_cluster.cc:1428] Setting key 663a3ce71ef9eb2df5187d78f256083b
I20260812 03:11:13.377754 11497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a: Bootstrap starting.
I20260812 03:11:13.380820 11497 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a: Neither blocks nor log segments found. Creating new log.
I20260812 03:11:13.381935 11497 log.cc:824] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a: Log is configured to *not* fsync() on all Append() calls
Aug 12 03:11:13 dist-test-slave-zpfg krb5kdc[11413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:11:13.384559 11497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a: No bootstrap required, opened a new log
I20260812 03:11:13.388619 11497 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7ac59b76224979a06891a373d00f2a" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 44865 } }
I20260812 03:11:13.388921 11497 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:11:13.388979 11497 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f7ac59b76224979a06891a373d00f2a, State: Initialized, Role: FOLLOWER
I20260812 03:11:13.389474 11497 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7ac59b76224979a06891a373d00f2a" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 44865 } }
I20260812 03:11:13.389612 11497 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:11:13.389701 11497 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:11:13.389822 11497 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:11:13.390617 11500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:13.373993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49216 (local address 127.26.250.190:44865)
0812 03:11:13.374587 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0812 03:11:13.374604 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:11:13.374646 (+ 42us) server_negotiation.cc:407] Connection header received
0812 03:11:13.375441 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:11:13.375462 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:11:13.376031 (+ 569us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:11:13.376510 (+ 479us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:11:13.377569 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:13.379146 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:13.380402 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:13.381042 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:13.381302 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:13.384471 (+ 3169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:11:13.384477 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:11:13.384493 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:11:13.384525 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:11:13.386660 (+ 2135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:11:13.387201 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:11:13.387206 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:11:13.387212 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:11:13.387340 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:11:13.387691 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:11:13.387694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:11:13.387698 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:11:13.388172 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:11:13.388470 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0812 03:11:13.388726 (+ 256us) server_negotiation.cc:299] Negotiation successful
0812 03:11:13.389104 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":169,"threads_started":1}
I20260812 03:11:13.391433 11497 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7ac59b76224979a06891a373d00f2a" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 44865 } }
I20260812 03:11:13.391834 11497 leader_election.cc:302] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f7ac59b76224979a06891a373d00f2a; no voters:
I20260812 03:11:13.392222 11497 leader_election.cc:288] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:11:13.392395 11502 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:11:13.392656 11502 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [term 1 LEADER]: Becoming Leader. State: Replica: 8f7ac59b76224979a06891a373d00f2a, State: Running, Role: LEADER
I20260812 03:11:13.393046 11502 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7ac59b76224979a06891a373d00f2a" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 44865 } }
I20260812 03:11:13.393652 11497 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:11:13.394982 11503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f7ac59b76224979a06891a373d00f2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7ac59b76224979a06891a373d00f2a" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 44865 } } }
I20260812 03:11:13.395165 11503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [sys.catalog]: This master's current role is: LEADER
I20260812 03:11:13.395610 11511 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:11:13.395574 11504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f7ac59b76224979a06891a373d00f2a. Latest consensus state: current_term: 1 leader_uuid: "8f7ac59b76224979a06891a373d00f2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7ac59b76224979a06891a373d00f2a" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 44865 } } }
I20260812 03:11:13.395754 11504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [sys.catalog]: This master's current role is: LEADER
I20260812 03:11:13.399752 11511 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:11:13.406490 11511 catalog_manager.cc:1347] Generated new cluster ID: c4438346609941f4befa8512f1269eb2
I20260812 03:11:13.406589 11511 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:11:13.419694 11511 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:11:13.421471 11511 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:11:13.428047 11511 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a: Generated new TSK 0
I20260812 03:11:13.428961 11511 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.250.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
I20260812 03:11:13.466933 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:0
--local_ip_for_outbound_sockets=127.26.250.129
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:44865
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:40161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:11:13.597149 11523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:11:13.597455 11523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:11:13.597501 11523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:11:13.601639 11523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:11:13.601711 11523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:11:13.601780 11523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:11:13.607585 11523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:40161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:44865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:11:13.608738 11523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:11:13.610067 11523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:11:13.620743 11531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:11:13.620759 11528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:11:13.620721 11529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:11:13.621088 11523 server_base.cc:1034] running on GCE node
I20260812 03:11:13.621610 11523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:11:13.622316 11523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:11:13.623549 11523 hybrid_clock.cc:648] HybridClock initialized: now 1786504273623523 us; error 45 us; skew 500 ppm
Aug 12 03:11:13 dist-test-slave-zpfg krb5kdc[11413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504273, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:11:13.626868 11523 init.cc:377] Logged in from keytab as kudu/127.26.250.129@KRBTEST.COM (short username kudu)
I20260812 03:11:13.627925 11523 webserver.cc:460] Webserver started at http://127.26.250.129:45551/ using document root <none> and password file <none>
I20260812 03:11:13.628567 11523 fs_manager.cc:359] Metadata directory not provided
I20260812 03:11:13.628669 11523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:11:13.628918 11523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:11:13.631251 11523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/data/instance:
uuid: "3754a4ccf1194f39bd37a10b93badcd4"
format_stamp: "Formatted at 2026-08-12 03:11:13 on dist-test-slave-zpfg"
server_key: "3aa1b0adc55e07c53302d5775c1836ea"
server_key_iv: "981d9ad35af1a2cf103e287ac27a0b85"
server_key_version: "encryptionkey@0"
I20260812 03:11:13.631906 11523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance:
uuid: "3754a4ccf1194f39bd37a10b93badcd4"
format_stamp: "Formatted at 2026-08-12 03:11:13 on dist-test-slave-zpfg"
server_key: "3aa1b0adc55e07c53302d5775c1836ea"
server_key_iv: "981d9ad35af1a2cf103e287ac27a0b85"
server_key_version: "encryptionkey@0"
I20260812 03:11:13.636382 11523 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:11:13.639492 11538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:11:13.641177 11523 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:11:13.641436 11523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "3754a4ccf1194f39bd37a10b93badcd4"
format_stamp: "Formatted at 2026-08-12 03:11:13 on dist-test-slave-zpfg"
server_key: "3aa1b0adc55e07c53302d5775c1836ea"
server_key_iv: "981d9ad35af1a2cf103e287ac27a0b85"
server_key_version: "encryptionkey@0"
I20260812 03:11:13.641595 11523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:11:13.658768 11523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:11:13.662569 11523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:11:13.662871 11523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:11:13.663614 11523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:11:13.664613 11523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:11:13.664667 11523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:11:13.664753 11523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:11:13.664789 11523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:11:13.675631 11523 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:43461
I20260812 03:11:13.675683 11661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:43461 every 8 connection(s)
I20260812 03:11:13.676963 11523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:11:13.686254 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 11523
I20260812 03:11:13.686444 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance
I20260812 03:11:13.686771 27626 external_mini_cluster.cc:1428] Setting key 108b9a87ef742def1928ff5d76321cc0
Aug 12 03:11:13 dist-test-slave-zpfg krb5kdc[11413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504273, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:11:13.693070 11500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:13.679296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:52855 (local address 127.26.250.190:44865)
0812 03:11:13.679451 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 03:11:13.679455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:11:13.680775 (+ 1320us) server_negotiation.cc:407] Connection header received
0812 03:11:13.681823 (+ 1048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:11:13.681828 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:11:13.681898 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:11:13.682019 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:11:13.683387 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:13.683947 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:13.685195 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:13.685439 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:13.685568 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:13.688761 (+ 3193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:11:13.688769 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:11:13.688771 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:11:13.688796 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:11:13.690483 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:11:13.691162 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:11:13.691168 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:11:13.691169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:11:13.691265 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:11:13.691748 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:11:13.691753 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:11:13.691755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:11:13.691979 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:11:13.692109 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 03:11:13.692738 (+ 629us) server_negotiation.cc:299] Negotiation successful
0812 03:11:13.692883 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 03:11:13.694162 11664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:13.679753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.129:52855)
0812 03:11:13.680584 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:13.680634 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 03:11:13.681558 (+ 924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:11:13.682202 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:11:13.682209 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:11:13.682691 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:11:13.683226 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:13.683240 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:13.684094 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:13.684102 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:13.685041 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:13.685051 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:13.685620 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:13.685625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:13.685809 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:13.686644 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:11:13.686663 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:11:13.688565 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:11:13.690620 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:11:13.690627 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:11:13.690646 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:11:13.690963 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:11:13.691458 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:11:13.691464 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:11:13.691466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:11:13.691638 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:11:13.692118 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:11:13.692151 (+ 33us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:11:13.692455 (+ 304us) client_negotiation.cc:769] Sending connection context
0812 03:11:13.692692 (+ 237us) client_negotiation.cc:240] Negotiation successful
0812 03:11:13.693094 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":149,"threads_started":1}
I20260812 03:11:13.695938 11662 heartbeater.cc:347] Connected to a master server at 127.26.250.190:44865
I20260812 03:11:13.696372 11662 heartbeater.cc:464] Registering TS with master...
I20260812 03:11:13.696970 11662 heartbeater.cc:511] Master 127.26.250.190:44865 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.26.250.130@KRBTEST.COM; defaulting to no policy
I20260812 03:11:13.698889 11458 ts_manager.cc:194] Registered new tserver with Master: 3754a4ccf1194f39bd37a10b93badcd4 (127.26.250.129:43461)
I20260812 03:11:13.700385 11458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.129@KRBTEST.COM'} at 127.26.250.129:52855
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
I20260812 03:11:13.723253 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:0
--local_ip_for_outbound_sockets=127.26.250.130
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:44865
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:40161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:11:13.849689 11667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:11:13.849994 11667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:11:13.850039 11667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:11:13.854297 11667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:11:13.854456 11667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:11:13.854566 11667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:11:13.860494 11667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:40161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:44865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:11:13.861720 11667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:11:13.862951 11667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:11:13.873852 11673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:11:13.873907 11675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:11:13.873924 11672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:11:13.874174 11667 server_base.cc:1034] running on GCE node
I20260812 03:11:13.874755 11667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:11:13.875578 11667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:11:13.876812 11667 hybrid_clock.cc:648] HybridClock initialized: now 1786504273876787 us; error 40 us; skew 500 ppm
Aug 12 03:11:13 dist-test-slave-zpfg krb5kdc[11413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504273, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:11:13.880539 11667 init.cc:377] Logged in from keytab as kudu/127.26.250.130@KRBTEST.COM (short username kudu)
I20260812 03:11:13.881760 11667 webserver.cc:460] Webserver started at http://127.26.250.130:35029/ using document root <none> and password file <none>
I20260812 03:11:13.882485 11667 fs_manager.cc:359] Metadata directory not provided
I20260812 03:11:13.882591 11667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:11:13.882854 11667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:11:13.885095 11667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/data/instance:
uuid: "24da118f679041f7b497d7d91c1cd831"
format_stamp: "Formatted at 2026-08-12 03:11:13 on dist-test-slave-zpfg"
server_key: "f3fb148c85edec6f70e7a0b0dd9f49e3"
server_key_iv: "11ce5d20e7f36ce904362b0fd6b3bde4"
server_key_version: "encryptionkey@0"
I20260812 03:11:13.885739 11667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance:
uuid: "24da118f679041f7b497d7d91c1cd831"
format_stamp: "Formatted at 2026-08-12 03:11:13 on dist-test-slave-zpfg"
server_key: "f3fb148c85edec6f70e7a0b0dd9f49e3"
server_key_iv: "11ce5d20e7f36ce904362b0fd6b3bde4"
server_key_version: "encryptionkey@0"
I20260812 03:11:13.889995 11667 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:11:13.893009 11683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:11:13.894577 11667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 03:11:13.894747 11667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "24da118f679041f7b497d7d91c1cd831"
format_stamp: "Formatted at 2026-08-12 03:11:13 on dist-test-slave-zpfg"
server_key: "f3fb148c85edec6f70e7a0b0dd9f49e3"
server_key_iv: "11ce5d20e7f36ce904362b0fd6b3bde4"
server_key_version: "encryptionkey@0"
I20260812 03:11:13.894908 11667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:11:13.907605 11667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:11:13.911365 11667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:11:13.911602 11667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:11:13.912300 11667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:11:13.913220 11667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:11:13.913295 11667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:11:13.913367 11667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:11:13.913414 11667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:11:13.923411 11667 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:41033
I20260812 03:11:13.923485 11804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:41033 every 8 connection(s)
I20260812 03:11:13.924458 11667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:11:13.932195 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 11667
I20260812 03:11:13.932410 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance
I20260812 03:11:13.932714 27626 external_mini_cluster.cc:1428] Setting key d9d13ea6afc7c6455acd8a9af7b563c9
Aug 12 03:11:13 dist-test-slave-zpfg krb5kdc[11413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504273, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:11:13.941432 11500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:13.926594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:48869 (local address 127.26.250.190:44865)
0812 03:11:13.926794 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 03:11:13.926836 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0812 03:11:13.927895 (+ 1059us) server_negotiation.cc:407] Connection header received
0812 03:11:13.929524 (+ 1629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:11:13.929527 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:11:13.929577 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:11:13.929662 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:11:13.931327 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:13.931913 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:13.933523 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:13.933764 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:13.933879 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:13.936830 (+ 2951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:11:13.936837 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:11:13.936841 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:11:13.936881 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:11:13.938805 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:11:13.939393 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:11:13.939400 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:11:13.939405 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:11:13.939513 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:11:13.939913 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:11:13.939919 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:11:13.939924 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:11:13.940244 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:11:13.940364 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 03:11:13.940988 (+ 624us) server_negotiation.cc:299] Negotiation successful
0812 03:11:13.941165 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260812 03:11:13.942301 11807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:13.927049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.130:48869)
0812 03:11:13.927684 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:13.927757 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0812 03:11:13.929262 (+ 1505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:11:13.929943 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:11:13.929951 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:11:13.930541 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:11:13.931135 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:13.931151 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:13.932089 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:13.932093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:13.933376 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:13.933387 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:13.933947 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:13.933951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:13.934181 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:13.934977 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:11:13.934996 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:11:13.936610 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:11:13.938941 (+ 2331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:11:13.938947 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:11:13.938961 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:11:13.939258 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:11:13.939617 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:11:13.939620 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:11:13.939621 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:11:13.939781 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:11:13.940370 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:11:13.940376 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:11:13.940687 (+ 311us) client_negotiation.cc:769] Sending connection context
0812 03:11:13.940863 (+ 176us) client_negotiation.cc:240] Negotiation successful
0812 03:11:13.941159 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":137,"threads_started":1}
I20260812 03:11:13.943838 11805 heartbeater.cc:347] Connected to a master server at 127.26.250.190:44865
I20260812 03:11:13.944115 11805 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.26.250.131@KRBTEST.COM; defaulting to no policy
I20260812 03:11:13.944774 11805 heartbeater.cc:511] Master 127.26.250.190:44865 requested a full tablet report, sending...
I20260812 03:11:13.946313 11458 ts_manager.cc:194] Registered new tserver with Master: 24da118f679041f7b497d7d91c1cd831 (127.26.250.130:41033)
I20260812 03:11:13.947033 11458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.130@KRBTEST.COM'} at 127.26.250.130:48869
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
I20260812 03:11:13.969810 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:0
--local_ip_for_outbound_sockets=127.26.250.131
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:44865
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:40161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:11:14.100117 11811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:11:14.100432 11811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:11:14.100482 11811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:11:14.105270 11811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:11:14.105355 11811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:11:14.105422 11811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:11:14.111531 11811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:40161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:44865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:11:14.113222 11811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:11:14.114519 11811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:11:14.125735 11816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:11:14.125855 11817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:11:14.125867 11819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:11:14.126412 11811 server_base.cc:1034] running on GCE node
I20260812 03:11:14.126981 11811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:11:14.127776 11811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:11:14.129061 11811 hybrid_clock.cc:648] HybridClock initialized: now 1786504274129027 us; error 56 us; skew 500 ppm
Aug 12 03:11:14 dist-test-slave-zpfg krb5kdc[11413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504274, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:11:14.132364 11811 init.cc:377] Logged in from keytab as kudu/127.26.250.131@KRBTEST.COM (short username kudu)
I20260812 03:11:14.133517 11811 webserver.cc:460] Webserver started at http://127.26.250.131:41767/ using document root <none> and password file <none>
I20260812 03:11:14.134227 11811 fs_manager.cc:359] Metadata directory not provided
I20260812 03:11:14.134327 11811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:11:14.134586 11811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:11:14.136842 11811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/data/instance:
uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20"
format_stamp: "Formatted at 2026-08-12 03:11:14 on dist-test-slave-zpfg"
server_key: "77fce3b673be7368add2a0bb4ee68587"
server_key_iv: "601dc1cf6460930ee9aed83c38df19c2"
server_key_version: "encryptionkey@0"
I20260812 03:11:14.137511 11811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance:
uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20"
format_stamp: "Formatted at 2026-08-12 03:11:14 on dist-test-slave-zpfg"
server_key: "77fce3b673be7368add2a0bb4ee68587"
server_key_iv: "601dc1cf6460930ee9aed83c38df19c2"
server_key_version: "encryptionkey@0"
I20260812 03:11:14.141995 11811 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260812 03:11:14.145092 11826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:11:14.146870 11811 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:11:14.147048 11811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20"
format_stamp: "Formatted at 2026-08-12 03:11:14 on dist-test-slave-zpfg"
server_key: "77fce3b673be7368add2a0bb4ee68587"
server_key_iv: "601dc1cf6460930ee9aed83c38df19c2"
server_key_version: "encryptionkey@0"
I20260812 03:11:14.147238 11811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:11:14.163672 11811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:11:14.167716 11811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:11:14.167968 11811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:11:14.168695 11811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:11:14.169658 11811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:11:14.169742 11811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:11:14.169823 11811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:11:14.169883 11811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:11:14.180840 11811 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:41271
I20260812 03:11:14.180889 11948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:41271 every 8 connection(s)
I20260812 03:11:14.181979 11811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:11:14.189059 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 11811
I20260812 03:11:14.189185 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance
I20260812 03:11:14.189648 27626 external_mini_cluster.cc:1428] Setting key 5dd6c99c5994594287f88a9164ccafad
Aug 12 03:11:14 dist-test-slave-zpfg krb5kdc[11413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504274, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:11:14.198711 11500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:14.184258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:45997 (local address 127.26.250.190:44865)
0812 03:11:14.184473 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 03:11:14.184477 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:11:14.185709 (+ 1232us) server_negotiation.cc:407] Connection header received
0812 03:11:14.187021 (+ 1312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:11:14.187024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:11:14.187079 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:11:14.187163 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:11:14.188528 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.189211 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.190819 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.191081 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.191216 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:14.194334 (+ 3118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:11:14.194338 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:11:14.194341 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:11:14.194437 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:11:14.196259 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:11:14.196914 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:11:14.196917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:11:14.196918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:11:14.196999 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:11:14.197397 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:11:14.197400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:11:14.197402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:11:14.197648 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:11:14.197759 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 03:11:14.198431 (+ 672us) server_negotiation.cc:299] Negotiation successful
0812 03:11:14.198579 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260812 03:11:14.199815 11951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:14.184768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.131:45997)
0812 03:11:14.185535 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:14.185581 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 03:11:14.186791 (+ 1210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:11:14.187413 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:11:14.187421 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:11:14.187818 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:11:14.188357 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:14.188371 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.189468 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.189472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:14.190671 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:14.190683 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.191285 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.191290 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:14.191485 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:14.192214 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:11:14.192232 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:11:14.194082 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:11:14.196441 (+ 2359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:11:14.196448 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:11:14.196464 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:11:14.196783 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:11:14.197098 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:11:14.197101 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:11:14.197102 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:11:14.197268 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:11:14.197775 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:11:14.197781 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:11:14.198084 (+ 303us) client_negotiation.cc:769] Sending connection context
0812 03:11:14.198269 (+ 185us) client_negotiation.cc:240] Negotiation successful
0812 03:11:14.198689 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":139,"threads_started":1}
I20260812 03:11:14.201169 11949 heartbeater.cc:347] Connected to a master server at 127.26.250.190:44865
I20260812 03:11:14.201481 11949 heartbeater.cc:464] Registering TS with master...
I20260812 03:11:14.202080 11949 heartbeater.cc:511] Master 127.26.250.190:44865 requested a full tablet report, sending...
I20260812 03:11:14.203382 11457 ts_manager.cc:194] Registered new tserver with Master: 0f3fa6b5eb1d4f16b072fd0387f60a20 (127.26.250.131:41271)
I20260812 03:11:14.204092 11457 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.131@KRBTEST.COM'} at 127.26.250.131:45997
I20260812 03:11:14.204316 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:11:14.215408 11500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:14.206882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49230 (local address 127.26.250.190:44865)
0812 03:11:14.207011 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 03:11:14.207018 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:11:14.207042 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:11:14.207166 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:11:14.207169 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:11:14.207207 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:11:14.207283 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:11:14.208281 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.208841 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.209824 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.210063 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.210162 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:14.211836 (+ 1674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:11:14.211839 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:11:14.211841 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:11:14.211865 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:11:14.213375 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:11:14.213864 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:11:14.213866 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:11:14.213868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:11:14.213950 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:11:14.214214 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:11:14.214217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:11:14.214218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:11:14.214652 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:11:14.214764 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 03:11:14.215089 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 03:11:14.215278 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 03:11:14.219183 11457 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49230:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:11:14.222709 11457 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:11:14.239464 11966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:14.233754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35114 (local address 127.26.250.130:41033)
0812 03:11:14.234162 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0812 03:11:14.234175 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:11:14.234205 (+ 30us) server_negotiation.cc:407] Connection header received
0812 03:11:14.234274 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:11:14.234281 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:11:14.234466 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:11:14.234663 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:11:14.234970 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.236052 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.237967 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.238703 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.238842 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:14.239005 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 03:11:14.239168 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 03:11:14.239271 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":180,"threads_started":1}
I20260812 03:11:14.239892 11965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:14.233492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:43461 (local address 127.0.0.1:44086)
0812 03:11:14.234248 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:14.234263 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:11:14.234476 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:11:14.235258 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:11:14.235261 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:11:14.235281 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:11:14.235364 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:14.235370 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.236806 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.236809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:14.237952 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:14.237960 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.239254 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.239258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:14.239426 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:14.239486 (+ 60us) client_negotiation.cc:769] Sending connection context
0812 03:11:14.239569 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 03:11:14.239722 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"mutex_wait_us":47,"thread_start_us":103,"threads_started":1}
I20260812 03:11:14.240132 11961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:14.232869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:41033 (local address 127.0.0.1:35114)
0812 03:11:14.233819 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:14.233837 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:11:14.233947 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:11:14.234630 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:11:14.234637 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:11:14.234660 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:11:14.234838 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:14.234846 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.236257 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.236260 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:14.237818 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:14.237829 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.238841 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.238845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:14.238970 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:14.239011 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 03:11:14.239126 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 03:11:14.239854 (+ 728us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":866,"mutex_wait_us":35,"thread_start_us":121,"threads_started":1}
I20260812 03:11:14.240291 11963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:14.232573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:41271 (local address 127.0.0.1:44800)
0812 03:11:14.233676 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:14.233740 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 03:11:14.233893 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:11:14.234812 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:11:14.234816 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:11:14.234832 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:11:14.234969 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:14.234976 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.236686 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.236690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:14.237737 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:14.237747 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.239260 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.239263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:14.239423 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:14.239496 (+ 73us) client_negotiation.cc:769] Sending connection context
0812 03:11:14.239872 (+ 376us) client_negotiation.cc:240] Negotiation successful
0812 03:11:14.240147 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":901,"mutex_wait_us":57,"thread_start_us":112,"threads_started":1}
I20260812 03:11:14.240573 11967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:14.234184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44086 (local address 127.26.250.129:43461)
0812 03:11:14.234671 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0812 03:11:14.234679 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:11:14.234702 (+ 23us) server_negotiation.cc:407] Connection header received
0812 03:11:14.234856 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:11:14.234861 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:11:14.235109 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:11:14.235278 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:11:14.235463 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.236686 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.238073 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.238939 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.239054 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:14.239176 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 03:11:14.240261 (+ 1085us) server_negotiation.cc:299] Negotiation successful
0812 03:11:14.240394 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":113,"threads_started":1}
I20260812 03:11:14.240834 11962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:14.232731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44800 (local address 127.26.250.131:41271)
0812 03:11:14.233388 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0812 03:11:14.233401 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:11:14.233865 (+ 464us) server_negotiation.cc:407] Connection header received
0812 03:11:14.234112 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:11:14.234119 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:11:14.234441 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:11:14.234605 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:11:14.235110 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.236497 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.237876 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.238894 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.239041 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:14.239176 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 03:11:14.240529 (+ 1353us) server_negotiation.cc:299] Negotiation successful
0812 03:11:14.240650 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":254,"threads_started":1}
I20260812 03:11:14.243458 11591 tablet_service.cc:1467] Processing CreateTablet for tablet e1de009554da42d8a678e5c12cf8ffdf (DEFAULT_TABLE table=test-table [id=f975c182182c4a8ca63a0facd39cac90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:11:14.243608 11877 tablet_service.cc:1467] Processing CreateTablet for tablet e1de009554da42d8a678e5c12cf8ffdf (DEFAULT_TABLE table=test-table [id=f975c182182c4a8ca63a0facd39cac90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:11:14.243609 11734 tablet_service.cc:1467] Processing CreateTablet for tablet e1de009554da42d8a678e5c12cf8ffdf (DEFAULT_TABLE table=test-table [id=f975c182182c4a8ca63a0facd39cac90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:11:14.244572 11591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1de009554da42d8a678e5c12cf8ffdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:11:14.244730 11734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1de009554da42d8a678e5c12cf8ffdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:11:14.244736 11877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1de009554da42d8a678e5c12cf8ffdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:11:14.251277 11968 tablet_bootstrap.cc:492] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4: Bootstrap starting.
I20260812 03:11:14.252900 11970 tablet_bootstrap.cc:492] T e1de009554da42d8a678e5c12cf8ffdf P 24da118f679041f7b497d7d91c1cd831: Bootstrap starting.
I20260812 03:11:14.254655 11968 tablet_bootstrap.cc:654] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4: Neither blocks nor log segments found. Creating new log.
I20260812 03:11:14.255065 11969 tablet_bootstrap.cc:492] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20: Bootstrap starting.
I20260812 03:11:14.255673 11968 log.cc:824] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4: Log is configured to *not* fsync() on all Append() calls
I20260812 03:11:14.255728 11970 tablet_bootstrap.cc:654] T e1de009554da42d8a678e5c12cf8ffdf P 24da118f679041f7b497d7d91c1cd831: Neither blocks nor log segments found. Creating new log.
I20260812 03:11:14.256569 11970 log.cc:824] T e1de009554da42d8a678e5c12cf8ffdf P 24da118f679041f7b497d7d91c1cd831: Log is configured to *not* fsync() on all Append() calls
I20260812 03:11:14.257951 11969 tablet_bootstrap.cc:654] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20: Neither blocks nor log segments found. Creating new log.
I20260812 03:11:14.258132 11968 tablet_bootstrap.cc:492] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4: No bootstrap required, opened a new log
I20260812 03:11:14.258394 11968 ts_tablet_manager.cc:1397] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260812 03:11:14.258908 11969 log.cc:824] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20: Log is configured to *not* fsync() on all Append() calls
I20260812 03:11:14.258945 11970 tablet_bootstrap.cc:492] T e1de009554da42d8a678e5c12cf8ffdf P 24da118f679041f7b497d7d91c1cd831: No bootstrap required, opened a new log
I20260812 03:11:14.259157 11970 ts_tablet_manager.cc:1397] T e1de009554da42d8a678e5c12cf8ffdf P 24da118f679041f7b497d7d91c1cd831: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 03:11:14.260901 11969 tablet_bootstrap.cc:492] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20: No bootstrap required, opened a new log
I20260812 03:11:14.261107 11969 ts_tablet_manager.cc:1397] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 03:11:14.263504 11970 raft_consensus.cc:348] T e1de009554da42d8a678e5c12cf8ffdf P 24da118f679041f7b497d7d91c1cd831 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } }
I20260812 03:11:14.263504 11968 raft_consensus.cc:348] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } }
I20260812 03:11:14.263855 11968 raft_consensus.cc:374] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:11:14.263856 11970 raft_consensus.cc:374] T e1de009554da42d8a678e5c12cf8ffdf P 24da118f679041f7b497d7d91c1cd831 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:11:14.263933 11968 raft_consensus.cc:729] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3754a4ccf1194f39bd37a10b93badcd4, State: Initialized, Role: FOLLOWER
I20260812 03:11:14.263936 11970 raft_consensus.cc:729] T e1de009554da42d8a678e5c12cf8ffdf P 24da118f679041f7b497d7d91c1cd831 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24da118f679041f7b497d7d91c1cd831, State: Initialized, Role: FOLLOWER
I20260812 03:11:14.264465 11968 consensus_queue.cc:260] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } }
I20260812 03:11:14.264465 11970 consensus_queue.cc:260] T e1de009554da42d8a678e5c12cf8ffdf P 24da118f679041f7b497d7d91c1cd831 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } }
I20260812 03:11:14.265332 11805 heartbeater.cc:503] Master 127.26.250.190:44865 was elected leader, sending a full tablet report...
I20260812 03:11:14.265599 11662 heartbeater.cc:503] Master 127.26.250.190:44865 was elected leader, sending a full tablet report...
I20260812 03:11:14.265695 11968 ts_tablet_manager.cc:1428] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260812 03:11:14.266695 11970 ts_tablet_manager.cc:1428] T e1de009554da42d8a678e5c12cf8ffdf P 24da118f679041f7b497d7d91c1cd831: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 03:11:14.266919 11969 raft_consensus.cc:348] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } }
I20260812 03:11:14.267268 11969 raft_consensus.cc:374] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:11:14.267334 11969 raft_consensus.cc:729] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f3fa6b5eb1d4f16b072fd0387f60a20, State: Initialized, Role: FOLLOWER
I20260812 03:11:14.267867 11969 consensus_queue.cc:260] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } }
I20260812 03:11:14.268775 11949 heartbeater.cc:503] Master 127.26.250.190:44865 was elected leader, sending a full tablet report...
I20260812 03:11:14.269124 11969 ts_tablet_manager.cc:1428] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260812 03:11:14.278756 11976 raft_consensus.cc:482] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:11:14.278991 11976 raft_consensus.cc:504] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } }
I20260812 03:11:14.280275 11976 leader_election.cc:288] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3754a4ccf1194f39bd37a10b93badcd4 (127.26.250.129:43461), 24da118f679041f7b497d7d91c1cd831 (127.26.250.130:41033)
I20260812 03:11:14.285583 11967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:14.280781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:40485 (local address 127.26.250.129:43461)
0812 03:11:14.280915 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 03:11:14.280920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:11:14.280935 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:11:14.281096 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:11:14.281104 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:11:14.281160 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:11:14.281419 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:11:14.281891 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.282730 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.284328 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.284904 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.285032 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:14.285084 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 03:11:14.285344 (+ 260us) server_negotiation.cc:299] Negotiation successful
0812 03:11:14.285436 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 03:11:14.285632 11951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:14.280649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:43461 (local address 127.26.250.131:40485)
0812 03:11:14.280816 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:14.280835 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:11:14.280946 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:11:14.281431 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:11:14.281434 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:11:14.281453 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:11:14.281745 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:14.281757 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.282861 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.282863 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:14.284181 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:14.284192 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.285074 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.285077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:14.285182 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:14.285205 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:11:14.285275 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 03:11:14.285356 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 03:11:14.286121 11966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:14.280905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:37875 (local address 127.26.250.130:41033)
0812 03:11:14.281061 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 03:11:14.281066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:11:14.281789 (+ 723us) server_negotiation.cc:407] Connection header received
0812 03:11:14.281987 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:11:14.281991 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:11:14.282046 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:11:14.282132 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:11:14.282574 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.283192 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.284721 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.285332 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.285454 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:14.285499 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 03:11:14.285887 (+ 388us) server_negotiation.cc:299] Negotiation successful
0812 03:11:14.285988 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:11:14.286563 11977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:14.281129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:41033 (local address 127.26.250.131:37875)
0812 03:11:14.281668 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:14.281684 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:11:14.281864 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:11:14.282179 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:11:14.282189 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:11:14.282207 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:11:14.282439 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:14.282457 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.283332 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.283335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:14.284534 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:14.284543 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.285573 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.285577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:14.285684 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:14.285726 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 03:11:14.286327 (+ 601us) client_negotiation.cc:240] Negotiation successful
0812 03:11:14.286449 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":142,"threads_started":1}
I20260812 03:11:14.286549 11612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1de009554da42d8a678e5c12cf8ffdf" candidate_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3754a4ccf1194f39bd37a10b93badcd4" is_pre_election: true
I20260812 03:11:14.286903 11612 raft_consensus.cc:2454] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f3fa6b5eb1d4f16b072fd0387f60a20 in term 0.
I20260812 03:11:14.287356 11758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1de009554da42d8a678e5c12cf8ffdf" candidate_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24da118f679041f7b497d7d91c1cd831" is_pre_election: true
I20260812 03:11:14.287588 11828 leader_election.cc:302] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f3fa6b5eb1d4f16b072fd0387f60a20, 3754a4ccf1194f39bd37a10b93badcd4; no voters:
I20260812 03:11:14.287712 11758 raft_consensus.cc:2454] T e1de009554da42d8a678e5c12cf8ffdf P 24da118f679041f7b497d7d91c1cd831 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f3fa6b5eb1d4f16b072fd0387f60a20 in term 0.
I20260812 03:11:14.287935 11976 raft_consensus.cc:2781] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:11:14.288079 11976 raft_consensus.cc:482] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:11:14.288144 11976 raft_consensus.cc:3037] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:11:14.289342 11976 raft_consensus.cc:504] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } }
I20260812 03:11:14.289749 11976 leader_election.cc:288] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20 [CANDIDATE]: Term 1 election: Requested vote from peers 3754a4ccf1194f39bd37a10b93badcd4 (127.26.250.129:43461), 24da118f679041f7b497d7d91c1cd831 (127.26.250.130:41033)
I20260812 03:11:14.290263 11758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1de009554da42d8a678e5c12cf8ffdf" candidate_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24da118f679041f7b497d7d91c1cd831"
I20260812 03:11:14.290462 11758 raft_consensus.cc:3037] T e1de009554da42d8a678e5c12cf8ffdf P 24da118f679041f7b497d7d91c1cd831 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:11:14.290580 11612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1de009554da42d8a678e5c12cf8ffdf" candidate_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3754a4ccf1194f39bd37a10b93badcd4"
I20260812 03:11:14.290737 11612 raft_consensus.cc:3037] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:11:14.292261 11758 raft_consensus.cc:2454] T e1de009554da42d8a678e5c12cf8ffdf P 24da118f679041f7b497d7d91c1cd831 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f3fa6b5eb1d4f16b072fd0387f60a20 in term 1.
I20260812 03:11:14.292399 11612 raft_consensus.cc:2454] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f3fa6b5eb1d4f16b072fd0387f60a20 in term 1.
I20260812 03:11:14.292766 11828 leader_election.cc:302] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0f3fa6b5eb1d4f16b072fd0387f60a20, 24da118f679041f7b497d7d91c1cd831, 3754a4ccf1194f39bd37a10b93badcd4; no voters:
I20260812 03:11:14.293084 11976 raft_consensus.cc:2781] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:11:14.293411 11976 raft_consensus.cc:686] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20 [term 1 LEADER]: Becoming Leader. State: Replica: 0f3fa6b5eb1d4f16b072fd0387f60a20, State: Running, Role: LEADER
I20260812 03:11:14.293828 11976 consensus_queue.cc:237] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } }
I20260812 03:11:14.297749 11455 catalog_manager.cc:5495] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f3fa6b5eb1d4f16b072fd0387f60a20 (127.26.250.131). New cstate: current_term: 1 leader_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } health_report { overall_health: UNKNOWN } } }
I20260812 03:11:14.320881 11962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:14.317010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44808 (local address 127.26.250.131:41271)
0812 03:11:14.317144 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 03:11:14.317165 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 03:11:14.317181 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:11:14.317264 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:11:14.317266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:11:14.317326 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:11:14.317432 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:11:14.317743 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.318275 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.319444 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:14.319658 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:14.319740 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:14.320077 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:11:14.320526 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:11:14.320619 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 03:11:14.320703 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 03:11:14.320760 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:11:14.328231 11612 raft_consensus.cc:1261] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4 [term 1 FOLLOWER]: Refusing update from remote peer 0f3fa6b5eb1d4f16b072fd0387f60a20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:11:14.328231 11758 raft_consensus.cc:1261] T e1de009554da42d8a678e5c12cf8ffdf P 24da118f679041f7b497d7d91c1cd831 [term 1 FOLLOWER]: Refusing update from remote peer 0f3fa6b5eb1d4f16b072fd0387f60a20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:11:14.329061 11976 consensus_queue.cc:1035] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:11:14.329301 11978 consensus_queue.cc:1035] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:11:14.338765 11982 mvcc.cc:205] Tried to move back new op lower bound from 7317521507639398400 to 7317521507511664640. Current Snapshot: MvccSnapshot[applied={T|T < 7317521507639398400}]
I20260812 03:11:14.347019 11455 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:11:14.347236 11455 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:11:14.350180 11455 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a: Sending DeleteTablet for 3 replicas of tablet e1de009554da42d8a678e5c12cf8ffdf
I20260812 03:11:14.351325 11877 tablet_service.cc:1514] Processing DeleteTablet for tablet e1de009554da42d8a678e5c12cf8ffdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:11:14 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:44800
I20260812 03:11:14.351433 11734 tablet_service.cc:1514] Processing DeleteTablet for tablet e1de009554da42d8a678e5c12cf8ffdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:11:14 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:35114
I20260812 03:11:14.351455 11591 tablet_service.cc:1514] Processing DeleteTablet for tablet e1de009554da42d8a678e5c12cf8ffdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:11:14 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:44086
I20260812 03:11:14.352035 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 11523
I20260812 03:11:14.354445 11990 tablet_replica.cc:330] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20: stopping tablet replica
I20260812 03:11:14.354931 11990 raft_consensus.cc:2229] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:11:14.355412 11990 raft_consensus.cc:2258] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:11:14.357357 11992 tablet_replica.cc:330] T e1de009554da42d8a678e5c12cf8ffdf P 24da118f679041f7b497d7d91c1cd831: stopping tablet replica
I20260812 03:11:14.357718 11992 raft_consensus.cc:2229] T e1de009554da42d8a678e5c12cf8ffdf P 24da118f679041f7b497d7d91c1cd831 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:11:14.358083 11992 raft_consensus.cc:2258] T e1de009554da42d8a678e5c12cf8ffdf P 24da118f679041f7b497d7d91c1cd831 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:11:14.359722 11992 ts_tablet_manager.cc:1905] T e1de009554da42d8a678e5c12cf8ffdf P 24da118f679041f7b497d7d91c1cd831: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:11:14.362435 11992 ts_tablet_manager.cc:1918] T e1de009554da42d8a678e5c12cf8ffdf P 24da118f679041f7b497d7d91c1cd831: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:11:14.362583 11992 log.cc:1196] T e1de009554da42d8a678e5c12cf8ffdf P 24da118f679041f7b497d7d91c1cd831: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/wal/wals/e1de009554da42d8a678e5c12cf8ffdf
I20260812 03:11:14.363047 11992 ts_tablet_manager.cc:1939] T e1de009554da42d8a678e5c12cf8ffdf P 24da118f679041f7b497d7d91c1cd831: Deleting consensus metadata
I20260812 03:11:14.364154 11990 ts_tablet_manager.cc:1905] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:11:14.364590 11445 catalog_manager.cc:4841] TS 24da118f679041f7b497d7d91c1cd831 (127.26.250.130:41033): tablet e1de009554da42d8a678e5c12cf8ffdf (table test-table [id=f975c182182c4a8ca63a0facd39cac90]) successfully deleted
W20260812 03:11:14.366235 11443 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.250.129:43461 (error 108)
I20260812 03:11:14.367695 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 11667
I20260812 03:11:14.367722 11961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:14.367426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:43461 (local address 127.0.0.1:44088)
0812 03:11:14.367567 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:14.367644 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.129:43461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 03:11:14.373266 11990 ts_tablet_manager.cc:1918] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:11:14.373492 11990 log.cc:1196] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/e1de009554da42d8a678e5c12cf8ffdf
I20260812 03:11:14.374089 11990 ts_tablet_manager.cc:1939] T e1de009554da42d8a678e5c12cf8ffdf P 0f3fa6b5eb1d4f16b072fd0387f60a20: Deleting consensus metadata
W20260812 03:11:14.379415 11443 catalog_manager.cc:4584] TS 3754a4ccf1194f39bd37a10b93badcd4 (127.26.250.129:43461): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e1de009554da42d8a678e5c12cf8ffdf: Network error: Client connection negotiation failed: client connection to 127.26.250.129:43461: connect: Connection refused (error 111)
I20260812 03:11:14.379693 11443 catalog_manager.cc:4841] TS 0f3fa6b5eb1d4f16b072fd0387f60a20 (127.26.250.131:41271): tablet e1de009554da42d8a678e5c12cf8ffdf (table test-table [id=f975c182182c4a8ca63a0facd39cac90]) successfully deleted
I20260812 03:11:14.387569 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 11811
I20260812 03:11:14.397346 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 11426
I20260812 03:11:34.407241 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:44865
--webserver_interface=127.26.250.190
--webserver_port=39479
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:40161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:44865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:11:34.529827 11995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:11:34.530165 11995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:11:34.530210 11995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:11:34.534494 11995 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 03:11:34.534571 11995 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 03:11:34.534588 11995 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 03:11:34.534603 11995 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:11:34.534622 11995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:11:34.534641 11995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:11:34.534720 11995 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:11:34.534749 11995 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:11:34.540484 11995 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:40161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:44865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:44865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=39479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:11:34.541730 11995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:11:34.542946 11995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:11:34.558386 12004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:11:34.558370 12002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:11:34.558501 11995 server_base.cc:1034] running on GCE node
W20260812 03:11:34.558375 12001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:11:34.559387 11995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:11:34.560645 11995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:11:34.561821 11995 hybrid_clock.cc:648] HybridClock initialized: now 1786504294561796 us; error 39 us; skew 500 ppm
Aug 12 03:11:34 dist-test-slave-zpfg krb5kdc[11413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504294, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:11:34.564968 11995 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:11:34.566051 11995 webserver.cc:460] Webserver started at http://127.26.250.190:39479/ using document root <none> and password file <none>
I20260812 03:11:34.566740 11995 fs_manager.cc:359] Metadata directory not provided
I20260812 03:11:34.566843 11995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:11:34.571173 11995 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:11:34.573786 12011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:11:34.575244 11995 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:11:34.575412 11995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "8f7ac59b76224979a06891a373d00f2a"
format_stamp: "Formatted at 2026-08-12 03:11:13 on dist-test-slave-zpfg"
server_key: "4c1016cd34d3c107df325752d87c2211"
server_key_iv: "2e8a470e634bbb3118c8cc1e88db6735"
server_key_version: "encryptionkey@0"
I20260812 03:11:34.576225 11995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:11:34.590422 11995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:11:34.594671 11995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:11:34.594957 11995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:11:34.603188 11995 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:44865
I20260812 03:11:34.603507 12070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:44865 every 8 connection(s)
I20260812 03:11:34.604513 11995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:11:34.606487 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 11995
I20260812 03:11:34.606945 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:43461
--local_ip_for_outbound_sockets=127.26.250.129
--tserver_master_addrs=127.26.250.190:44865
--webserver_port=45551
--webserver_interface=127.26.250.129
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:40161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:11:34.609633 12071 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:11:34.611549 12071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a: Bootstrap starting.
I20260812 03:11:34.631115 12071 log.cc:824] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:11:34.640764 12071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:11:34.641217 12071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a: Bootstrap complete.
I20260812 03:11:34.645658 12071 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7ac59b76224979a06891a373d00f2a" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 44865 } }
I20260812 03:11:34.646126 12071 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f7ac59b76224979a06891a373d00f2a, State: Initialized, Role: FOLLOWER
I20260812 03:11:34.646720 12071 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7ac59b76224979a06891a373d00f2a" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 44865 } }
I20260812 03:11:34.646880 12071 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:11:34.646984 12071 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:11:34.647109 12071 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [term 1 FOLLOWER]: Advancing to term 2
I20260812 03:11:34.649542 12071 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7ac59b76224979a06891a373d00f2a" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 44865 } }
I20260812 03:11:34.649937 12071 leader_election.cc:302] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f7ac59b76224979a06891a373d00f2a; no voters:
I20260812 03:11:34.650336 12071 leader_election.cc:288] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 03:11:34.650494 12075 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [term 2 FOLLOWER]: Leader election won for term 2
I20260812 03:11:34.650709 12075 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [term 2 LEADER]: Becoming Leader. State: Replica: 8f7ac59b76224979a06891a373d00f2a, State: Running, Role: LEADER
I20260812 03:11:34.651119 12075 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7ac59b76224979a06891a373d00f2a" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 44865 } }
I20260812 03:11:34.651762 12071 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:11:34.652168 12075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f7ac59b76224979a06891a373d00f2a. Latest consensus state: current_term: 2 leader_uuid: "8f7ac59b76224979a06891a373d00f2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7ac59b76224979a06891a373d00f2a" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 44865 } } }
I20260812 03:11:34.652383 12075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [sys.catalog]: This master's current role is: LEADER
I20260812 03:11:34.652624 12076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8f7ac59b76224979a06891a373d00f2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7ac59b76224979a06891a373d00f2a" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 44865 } } }
I20260812 03:11:34.652740 12076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [sys.catalog]: This master's current role is: LEADER
I20260812 03:11:34.652882 12079 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:11:34.658294 12079 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:11:34.659260 12079 catalog_manager.cc:1259] Loaded cluster ID: c4438346609941f4befa8512f1269eb2
I20260812 03:11:34.659339 12079 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:11:34.662122 12079 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:11:34.662992 12079 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a: Loaded TSK: 0
I20260812 03:11:34.683211 12079 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a: Generated new TSK 1
I20260812 03:11:34.683553 12079 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:11:34.736663 12073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:11:34.736969 12073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:11:34.737016 12073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:11:34.741343 12073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:11:34.741429 12073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:11:34.741493 12073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:11:34.747454 12073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:40161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:43461
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=45551
--tserver_master_addrs=127.26.250.190:44865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:11:34.748734 12073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:11:34.749836 12073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:11:34.762056 12099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:11:34.762432 12073 server_base.cc:1034] running on GCE node
W20260812 03:11:34.762104 12100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:11:34.762116 12102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:11:34.763113 12073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:11:34.763945 12073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:11:34.765161 12073 hybrid_clock.cc:648] HybridClock initialized: now 1786504294765137 us; error 39 us; skew 500 ppm
Aug 12 03:11:34 dist-test-slave-zpfg krb5kdc[11413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504294, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:11:34.768692 12073 init.cc:377] Logged in from keytab as kudu/127.26.250.129@KRBTEST.COM (short username kudu)
I20260812 03:11:34.769994 12073 webserver.cc:460] Webserver started at http://127.26.250.129:45551/ using document root <none> and password file <none>
I20260812 03:11:34.770728 12073 fs_manager.cc:359] Metadata directory not provided
I20260812 03:11:34.770823 12073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:11:34.776654 12073 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260812 03:11:34.780375 12109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:11:34.782544 12073 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260812 03:11:34.782743 12073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "3754a4ccf1194f39bd37a10b93badcd4"
format_stamp: "Formatted at 2026-08-12 03:11:13 on dist-test-slave-zpfg"
server_key: "3aa1b0adc55e07c53302d5775c1836ea"
server_key_iv: "981d9ad35af1a2cf103e287ac27a0b85"
server_key_version: "encryptionkey@0"
I20260812 03:11:34.783639 12073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:11:34.800822 12073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:11:34.805119 12073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:11:34.805368 12073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:11:34.806135 12073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:11:34.807453 12116 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 03:11:34.813277 12073 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 03:11:34.813508 12073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260812 03:11:34.813617 12073 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 03:11:34.815734 12073 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 03:11:34.815830 12073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260812 03:11:34.816226 12116 tablet_bootstrap.cc:492] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4: Bootstrap starting.
I20260812 03:11:34.829224 12073 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:43461
I20260812 03:11:34.829317 12227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:43461 every 8 connection(s)
I20260812 03:11:34.830473 12073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:11:34.836717 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 12073
I20260812 03:11:34.837257 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:41033
--local_ip_for_outbound_sockets=127.26.250.130
--tserver_master_addrs=127.26.250.190:44865
--webserver_port=35029
--webserver_interface=127.26.250.130
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:40161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:11:34.840184 12116 log.cc:824] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4: Log is configured to *not* fsync() on all Append() calls
Aug 12 03:11:34 dist-test-slave-zpfg krb5kdc[11413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504294, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:11:34.851125 12116 tablet_bootstrap.cc:492] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:11:34.851421 12231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:34.833018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:43677 (local address 127.26.250.190:44865)
0812 03:11:34.833513 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0812 03:11:34.833537 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 03:11:34.834015 (+ 478us) server_negotiation.cc:407] Connection header received
0812 03:11:34.835577 (+ 1562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:11:34.835590 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:11:34.835957 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:11:34.836410 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:11:34.838038 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:34.839783 (+ 1745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:34.841473 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:34.842017 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:34.842231 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:34.845223 (+ 2992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:11:34.845229 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:11:34.845243 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:11:34.845278 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:11:34.847589 (+ 2311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:11:34.848227 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:11:34.848232 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:11:34.848237 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:11:34.848451 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:11:34.848880 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:11:34.848884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:11:34.848886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:11:34.849295 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:11:34.849463 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 03:11:34.850059 (+ 596us) server_negotiation.cc:299] Negotiation successful
0812 03:11:34.850386 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":149,"threads_started":1}
I20260812 03:11:34.851727 12116 tablet_bootstrap.cc:492] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4: Bootstrap complete.
I20260812 03:11:34.852077 12232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:34.833026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.129:43677)
0812 03:11:34.833769 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:34.833850 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0812 03:11:34.835052 (+ 1202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:11:34.836442 (+ 1390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:11:34.836453 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:11:34.837079 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:11:34.837806 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:34.837829 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:34.839991 (+ 2162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:34.839994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:34.841301 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:34.841326 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:34.842204 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:34.842208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:34.842469 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:34.843237 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:11:34.843256 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:11:34.844972 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:11:34.847741 (+ 2769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:11:34.847747 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:11:34.847762 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:11:34.848071 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:11:34.848584 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:11:34.848588 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:11:34.848591 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:11:34.848759 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:11:34.849432 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:11:34.849439 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:11:34.849714 (+ 275us) client_negotiation.cc:769] Sending connection context
0812 03:11:34.849926 (+ 212us) client_negotiation.cc:240] Negotiation successful
0812 03:11:34.850373 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":322,"threads_started":1}
I20260812 03:11:34.854534 12116 ts_tablet_manager.cc:1397] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4: Time spent bootstrapping tablet: real 0.039s user 0.029s sys 0.007s
I20260812 03:11:34.854892 12228 heartbeater.cc:347] Connected to a master server at 127.26.250.190:44865
I20260812 03:11:34.855159 12228 heartbeater.cc:464] Registering TS with master...
I20260812 03:11:34.855715 12228 heartbeater.cc:511] Master 127.26.250.190:44865 requested a full tablet report, sending...
I20260812 03:11:34.858599 12029 ts_manager.cc:194] Registered new tserver with Master: 3754a4ccf1194f39bd37a10b93badcd4 (127.26.250.129:43461)
I20260812 03:11:34.858625 12116 raft_consensus.cc:348] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } }
I20260812 03:11:34.859195 12116 raft_consensus.cc:729] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3754a4ccf1194f39bd37a10b93badcd4, State: Initialized, Role: FOLLOWER
I20260812 03:11:34.859721 12116 consensus_queue.cc:260] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } }
I20260812 03:11:34.860736 12116 ts_tablet_manager.cc:1428] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4: Time spent starting tablet: real 0.006s user 0.003s sys 0.001s
I20260812 03:11:34.862926 12029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.129@KRBTEST.COM'} at 127.26.250.129:43677
I20260812 03:11:34.866545 12228 heartbeater.cc:503] Master 127.26.250.190:44865 was elected leader, sending a full tablet report...
I20260812 03:11:34.876380 12238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:34.862307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54840 (local address 127.26.250.129:43461)
0812 03:11:34.871250 (+ 8943us) server_negotiation.cc:206] Beginning negotiation
0812 03:11:34.871262 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:11:34.871287 (+ 25us) server_negotiation.cc:407] Connection header received
0812 03:11:34.871359 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:11:34.871364 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:11:34.871568 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:11:34.871734 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:11:34.872076 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:34.873234 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:34.874819 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:34.875608 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:34.875756 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:34.875894 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0812 03:11:34.876097 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 03:11:34.876221 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8827,"thread_start_us":175,"threads_started":1}
I20260812 03:11:34.876470 12237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:34.862261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:43461 (local address 127.0.0.1:54840)
0812 03:11:34.870716 (+ 8455us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:34.870783 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0812 03:11:34.870934 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:11:34.871770 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:11:34.871775 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:11:34.871802 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:11:34.871945 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:34.871951 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:34.873401 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:34.873404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:34.874670 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:34.874684 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:34.875774 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:34.875778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:34.875884 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:34.875921 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 03:11:34.876044 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 03:11:34.876258 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8249,"thread_start_us":199,"threads_started":1}
I20260812 03:11:34.877767 12159 tablet_service.cc:1514] Processing DeleteTablet for tablet e1de009554da42d8a678e5c12cf8ffdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:11:14 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:54840
I20260812 03:11:34.878847 12239 tablet_replica.cc:330] stopping tablet replica
I20260812 03:11:34.879194 12239 raft_consensus.cc:2229] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:11:34.879529 12239 raft_consensus.cc:2258] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:11:34.880657 12239 ts_tablet_manager.cc:1905] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:11:34.885157 12239 ts_tablet_manager.cc:1918] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:11:34.885377 12239 log.cc:1196] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/wal/wals/e1de009554da42d8a678e5c12cf8ffdf
I20260812 03:11:34.885730 12239 ts_tablet_manager.cc:1939] T e1de009554da42d8a678e5c12cf8ffdf P 3754a4ccf1194f39bd37a10b93badcd4: Deleting consensus metadata
I20260812 03:11:34.887763 12013 catalog_manager.cc:4841] TS 3754a4ccf1194f39bd37a10b93badcd4 (127.26.250.129:43461): tablet e1de009554da42d8a678e5c12cf8ffdf (table test-table [id=f975c182182c4a8ca63a0facd39cac90]) successfully deleted
W20260812 03:11:34.972875 12233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:11:34.973212 12233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:11:34.973309 12233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:11:34.977635 12233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:11:34.977741 12233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:11:34.977834 12233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:11:34.983639 12233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:40161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:41033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=35029
--tserver_master_addrs=127.26.250.190:44865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:11:34.984850 12233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:11:34.985965 12233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:11:34.997274 12244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:11:34.997371 12233 server_base.cc:1034] running on GCE node
W20260812 03:11:34.997277 12245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:11:34.997282 12247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:11:34.997964 12233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:11:34.998798 12233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:11:35.000006 12233 hybrid_clock.cc:648] HybridClock initialized: now 1786504294999983 us; error 48 us; skew 500 ppm
Aug 12 03:11:35 dist-test-slave-zpfg krb5kdc[11413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504295, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:11:35.003372 12233 init.cc:377] Logged in from keytab as kudu/127.26.250.130@KRBTEST.COM (short username kudu)
I20260812 03:11:35.004437 12233 webserver.cc:460] Webserver started at http://127.26.250.130:35029/ using document root <none> and password file <none>
I20260812 03:11:35.005117 12233 fs_manager.cc:359] Metadata directory not provided
I20260812 03:11:35.005240 12233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:11:35.009613 12233 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:11:35.011911 12254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:11:35.013329 12233 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:11:35.013481 12233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "24da118f679041f7b497d7d91c1cd831"
format_stamp: "Formatted at 2026-08-12 03:11:13 on dist-test-slave-zpfg"
server_key: "f3fb148c85edec6f70e7a0b0dd9f49e3"
server_key_iv: "11ce5d20e7f36ce904362b0fd6b3bde4"
server_key_version: "encryptionkey@0"
I20260812 03:11:35.014072 12233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:11:35.039361 12233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:11:35.043048 12233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:11:35.043300 12233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:11:35.044051 12233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:11:35.044974 12233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:11:35.045050 12233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:11:35.045125 12233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:11:35.045171 12233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:11:35.056156 12233 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:41033
I20260812 03:11:35.056208 12375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:41033 every 8 connection(s)
I20260812 03:11:35.057286 12233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:11:35.067358 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 12233
I20260812 03:11:35.067916 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:41271
--local_ip_for_outbound_sockets=127.26.250.131
--tserver_master_addrs=127.26.250.190:44865
--webserver_port=41767
--webserver_interface=127.26.250.131
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:40161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 03:11:35 dist-test-slave-zpfg krb5kdc[11413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504295, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:11:35.074066 12231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:35.059805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:56457 (local address 127.26.250.190:44865)
0812 03:11:35.059990 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 03:11:35.059994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:11:35.060904 (+ 910us) server_negotiation.cc:407] Connection header received
0812 03:11:35.062496 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:11:35.062500 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:11:35.062569 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:11:35.062686 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:11:35.064069 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.064637 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.065836 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.066084 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.066200 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:35.069194 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:11:35.069197 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:11:35.069199 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:11:35.069225 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:11:35.071364 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:11:35.072171 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:11:35.072174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:11:35.072175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:11:35.072261 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:11:35.072690 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:11:35.072692 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:11:35.072693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:11:35.072886 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:11:35.072963 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 03:11:35.073693 (+ 730us) server_negotiation.cc:299] Negotiation successful
0812 03:11:35.073928 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 03:11:35.075393 12378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:35.060178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.130:56457)
0812 03:11:35.060733 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:35.060780 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:11:35.062214 (+ 1434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:11:35.062890 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:11:35.062898 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:11:35.063400 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:11:35.063906 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:35.063921 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.064785 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.064788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:35.065694 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:35.065703 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.066257 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.066261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:35.066466 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:35.067137 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:11:35.067160 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:11:35.068997 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:11:35.071536 (+ 2539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:11:35.071547 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:11:35.071569 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:11:35.072050 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:11:35.072369 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:11:35.072375 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:11:35.072380 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:11:35.072590 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:11:35.072997 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:11:35.073008 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:11:35.073396 (+ 388us) client_negotiation.cc:769] Sending connection context
0812 03:11:35.073654 (+ 258us) client_negotiation.cc:240] Negotiation successful
0812 03:11:35.074101 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":135,"threads_started":1}
I20260812 03:11:35.076956 12376 heartbeater.cc:347] Connected to a master server at 127.26.250.190:44865
I20260812 03:11:35.077333 12376 heartbeater.cc:464] Registering TS with master...
I20260812 03:11:35.077877 12376 heartbeater.cc:511] Master 127.26.250.190:44865 requested a full tablet report, sending...
I20260812 03:11:35.079160 12029 ts_manager.cc:194] Registered new tserver with Master: 24da118f679041f7b497d7d91c1cd831 (127.26.250.130:41033)
I20260812 03:11:35.080102 12029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.130@KRBTEST.COM'} at 127.26.250.130:56457
W20260812 03:11:35.195578 12379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:11:35.195923 12379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:11:35.195976 12379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:11:35.200826 12379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:11:35.200915 12379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:11:35.200979 12379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:11:35.207124 12379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:40161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:41271
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=41767
--tserver_master_addrs=127.26.250.190:44865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:11:35.208400 12379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:11:35.209520 12379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:11:35.220875 12387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:11:35.220862 12384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:11:35.221104 12379 server_base.cc:1034] running on GCE node
W20260812 03:11:35.220862 12385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:11:35.221647 12379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:11:35.222477 12379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:11:35.223663 12379 hybrid_clock.cc:648] HybridClock initialized: now 1786504295223645 us; error 34 us; skew 500 ppm
Aug 12 03:11:35 dist-test-slave-zpfg krb5kdc[11413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504295, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:11:35.231451 12379 init.cc:377] Logged in from keytab as kudu/127.26.250.131@KRBTEST.COM (short username kudu)
I20260812 03:11:35.232702 12379 webserver.cc:460] Webserver started at http://127.26.250.131:41767/ using document root <none> and password file <none>
I20260812 03:11:35.233460 12379 fs_manager.cc:359] Metadata directory not provided
I20260812 03:11:35.233623 12379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:11:35.238785 12379 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:11:35.241516 12395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:11:35.243252 12379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:11:35.243383 12379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20"
format_stamp: "Formatted at 2026-08-12 03:11:14 on dist-test-slave-zpfg"
server_key: "77fce3b673be7368add2a0bb4ee68587"
server_key_iv: "601dc1cf6460930ee9aed83c38df19c2"
server_key_version: "encryptionkey@0"
I20260812 03:11:35.244069 12379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:11:35.264891 12379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:11:35.268774 12379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:11:35.269023 12379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:11:35.269904 12379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:11:35.271173 12379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:11:35.271230 12379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:11:35.271278 12379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:11:35.271304 12379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:11:35.282382 12379 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:41271
I20260812 03:11:35.282543 12518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:41271 every 8 connection(s)
I20260812 03:11:35.283578 12379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:11:35.288090 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 12379
Aug 12 03:11:35 dist-test-slave-zpfg krb5kdc[11413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504295, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:11:35.298147 12523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:35.289013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48562 (local address 127.26.250.190:44865)
0812 03:11:35.289339 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0812 03:11:35.289344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:11:35.289360 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:11:35.289503 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:11:35.289506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:11:35.289557 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:11:35.289656 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:11:35.290510 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.291056 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.291947 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.292254 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.292457 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:35.293998 (+ 1541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:11:35.294001 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:11:35.294003 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:11:35.294029 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:11:35.296044 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:11:35.296692 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:11:35.296696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:11:35.296706 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:11:35.296809 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:11:35.297107 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:11:35.297111 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:11:35.297114 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:11:35.297357 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:11:35.297525 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 03:11:35.297810 (+ 285us) server_negotiation.cc:299] Negotiation successful
0812 03:11:35.297975 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":248,"thread_start_us":128,"threads_started":1}
I20260812 03:11:35.299606 12231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:35.285724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:59767 (local address 127.26.250.190:44865)
0812 03:11:35.285886 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 03:11:35.285890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:11:35.287052 (+ 1162us) server_negotiation.cc:407] Connection header received
0812 03:11:35.288151 (+ 1099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:11:35.288154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:11:35.288228 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:11:35.288338 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:11:35.290105 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.290725 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.291955 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.292237 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.292420 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:35.295307 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:11:35.295312 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:11:35.295315 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:11:35.295345 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:11:35.297095 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:11:35.297753 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:11:35.297757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:11:35.297758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:11:35.297848 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:11:35.298306 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:11:35.298309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:11:35.298311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:11:35.298577 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:11:35.298679 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 03:11:35.299314 (+ 635us) server_negotiation.cc:299] Negotiation successful
0812 03:11:35.299456 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":69,"spinlock_wait_cycles":4736}
I20260812 03:11:35.300719 12521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:35.286154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.131:59767)
0812 03:11:35.286883 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:35.286927 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:11:35.287928 (+ 1001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:11:35.288539 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:11:35.288570 (+ 31us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:11:35.289148 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:11:35.289945 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:35.289961 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.290854 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.290857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:35.291817 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:35.291827 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.292466 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.292470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:35.292669 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:35.293339 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:11:35.293356 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:11:35.295090 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:11:35.297217 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:11:35.297224 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:11:35.297245 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:11:35.297619 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:11:35.297977 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:11:35.297981 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:11:35.297983 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:11:35.298199 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:11:35.298702 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:11:35.298711 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:11:35.299050 (+ 339us) client_negotiation.cc:769] Sending connection context
0812 03:11:35.299271 (+ 221us) client_negotiation.cc:240] Negotiation successful
0812 03:11:35.299706 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":164,"threads_started":1}
I20260812 03:11:35.302044 12519 heartbeater.cc:347] Connected to a master server at 127.26.250.190:44865
I20260812 03:11:35.302373 12519 heartbeater.cc:464] Registering TS with master...
I20260812 03:11:35.302979 12519 heartbeater.cc:511] Master 127.26.250.190:44865 requested a full tablet report, sending...
I20260812 03:11:35.304147 12029 ts_manager.cc:194] Registered new tserver with Master: 0f3fa6b5eb1d4f16b072fd0387f60a20 (127.26.250.131:41271)
I20260812 03:11:35.304914 12029 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.131@KRBTEST.COM'} at 127.26.250.131:59767
I20260812 03:11:35.312973 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:11:35.318513 12231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:35.313747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48572 (local address 127.26.250.190:44865)
0812 03:11:35.313931 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 03:11:35.313936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:11:35.314218 (+ 282us) server_negotiation.cc:407] Connection header received
0812 03:11:35.314561 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:11:35.314565 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:11:35.314724 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:11:35.314860 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:11:35.315162 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.315809 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.316936 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.317203 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.317302 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:35.317659 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:11:35.317853 (+ 194us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 03:11:35.318297 (+ 444us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48572: token expired
Metrics: {"server-negotiator.queue_time_us":57}
W20260812 03:11:35.318615 12231 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48572: token expired
W20260812 03:11:35.318836 11955 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.26.250.190:44865)
I20260812 03:11:35.327327 12231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:35.319296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48574 (local address 127.26.250.190:44865)
0812 03:11:35.319504 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 03:11:35.319507 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:11:35.319521 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:11:35.319555 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:11:35.319557 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:11:35.319598 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:11:35.319664 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:11:35.320299 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.320901 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.321775 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.322022 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.322106 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:35.323678 (+ 1572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:11:35.323681 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:11:35.323683 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:11:35.323709 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:11:35.325380 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:11:35.325857 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:11:35.325860 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:11:35.325861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:11:35.325957 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:11:35.326239 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:11:35.326242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:11:35.326243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:11:35.326563 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:11:35.326679 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 03:11:35.326893 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 03:11:35.327002 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260812 03:11:35.330196 12029 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48574:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:11:35.334138 12029 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:11:35.343689 12159 tablet_service.cc:1467] Processing CreateTablet for tablet 642dce5bd0af4e959ca0608b334f9531 (DEFAULT_TABLE table=test-table [id=6f66120b797d4565be6c11a296d6a2db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:11:35.344426 12159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 642dce5bd0af4e959ca0608b334f9531. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:11:35.347415 12116 tablet_bootstrap.cc:492] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4: Bootstrap starting.
I20260812 03:11:35.348502 12116 tablet_bootstrap.cc:654] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4: Neither blocks nor log segments found. Creating new log.
I20260812 03:11:35.348805 12529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:35.342312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:41033 (local address 127.0.0.1:42632)
0812 03:11:35.342646 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:35.342660 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:11:35.342801 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:11:35.344026 (+ 1225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:11:35.344029 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:11:35.344046 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:11:35.344139 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:35.344144 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.345352 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.345358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:35.346832 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:35.346844 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.348283 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.348288 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:35.348416 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:35.348456 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 03:11:35.348535 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 03:11:35.348628 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":98,"thread_start_us":158,"threads_started":1}
I20260812 03:11:35.348927 12237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:35.342066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:41271 (local address 127.0.0.1:36526)
0812 03:11:35.342204 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:35.342221 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:11:35.342317 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:11:35.343342 (+ 1025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:11:35.343346 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:11:35.343366 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:11:35.343486 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:35.343492 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.345408 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.345411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:35.346832 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:35.346844 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.348471 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.348475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:35.348585 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:35.348609 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:11:35.348666 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 03:11:35.348756 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":165}
I20260812 03:11:35.348979 12531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:35.342925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42632 (local address 127.26.250.130:41033)
0812 03:11:35.343255 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0812 03:11:35.343271 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:11:35.343313 (+ 42us) server_negotiation.cc:407] Connection header received
0812 03:11:35.343393 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:11:35.343397 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:11:35.343604 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:11:35.343799 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:11:35.344263 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.345227 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.346995 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.348126 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.348300 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:35.348490 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 03:11:35.348564 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 03:11:35.348794 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":130,"threads_started":1}
I20260812 03:11:35.349903 12116 tablet_bootstrap.cc:492] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4: No bootstrap required, opened a new log
I20260812 03:11:35.350018 12116 ts_tablet_manager.cc:1397] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 03:11:35.350380 12530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:35.342294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36526 (local address 127.26.250.131:41271)
0812 03:11:35.342725 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0812 03:11:35.342742 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:11:35.342771 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:11:35.342860 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:11:35.342868 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:11:35.343116 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:11:35.343371 (+ 255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:11:35.343587 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.345258 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.346995 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.348243 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.349822 (+ 1579us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:35.349984 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 03:11:35.350068 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 03:11:35.350189 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":202,"threads_started":1}
I20260812 03:11:35.350482 12116 raft_consensus.cc:348] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } }
I20260812 03:11:35.350610 12116 raft_consensus.cc:374] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:11:35.350646 12116 raft_consensus.cc:729] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3754a4ccf1194f39bd37a10b93badcd4, State: Initialized, Role: FOLLOWER
I20260812 03:11:35.350781 12116 consensus_queue.cc:260] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } }
I20260812 03:11:35.351099 12116 ts_tablet_manager.cc:1428] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260812 03:11:35.352890 12306 tablet_service.cc:1467] Processing CreateTablet for tablet 642dce5bd0af4e959ca0608b334f9531 (DEFAULT_TABLE table=test-table [id=6f66120b797d4565be6c11a296d6a2db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:11:35.353461 12449 tablet_service.cc:1467] Processing CreateTablet for tablet 642dce5bd0af4e959ca0608b334f9531 (DEFAULT_TABLE table=test-table [id=6f66120b797d4565be6c11a296d6a2db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:11:35.354236 12306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 642dce5bd0af4e959ca0608b334f9531. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:11:35.354519 12449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 642dce5bd0af4e959ca0608b334f9531. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:11:35.362083 12533 tablet_bootstrap.cc:492] T 642dce5bd0af4e959ca0608b334f9531 P 0f3fa6b5eb1d4f16b072fd0387f60a20: Bootstrap starting.
I20260812 03:11:35.362262 12534 tablet_bootstrap.cc:492] T 642dce5bd0af4e959ca0608b334f9531 P 24da118f679041f7b497d7d91c1cd831: Bootstrap starting.
I20260812 03:11:35.364601 12534 tablet_bootstrap.cc:654] T 642dce5bd0af4e959ca0608b334f9531 P 24da118f679041f7b497d7d91c1cd831: Neither blocks nor log segments found. Creating new log.
I20260812 03:11:35.364630 12533 tablet_bootstrap.cc:654] T 642dce5bd0af4e959ca0608b334f9531 P 0f3fa6b5eb1d4f16b072fd0387f60a20: Neither blocks nor log segments found. Creating new log.
I20260812 03:11:35.365391 12533 log.cc:824] T 642dce5bd0af4e959ca0608b334f9531 P 0f3fa6b5eb1d4f16b072fd0387f60a20: Log is configured to *not* fsync() on all Append() calls
I20260812 03:11:35.365401 12534 log.cc:824] T 642dce5bd0af4e959ca0608b334f9531 P 24da118f679041f7b497d7d91c1cd831: Log is configured to *not* fsync() on all Append() calls
I20260812 03:11:35.367650 12533 tablet_bootstrap.cc:492] T 642dce5bd0af4e959ca0608b334f9531 P 0f3fa6b5eb1d4f16b072fd0387f60a20: No bootstrap required, opened a new log
I20260812 03:11:35.367648 12534 tablet_bootstrap.cc:492] T 642dce5bd0af4e959ca0608b334f9531 P 24da118f679041f7b497d7d91c1cd831: No bootstrap required, opened a new log
I20260812 03:11:35.367883 12533 ts_tablet_manager.cc:1397] T 642dce5bd0af4e959ca0608b334f9531 P 0f3fa6b5eb1d4f16b072fd0387f60a20: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:11:35.367887 12534 ts_tablet_manager.cc:1397] T 642dce5bd0af4e959ca0608b334f9531 P 24da118f679041f7b497d7d91c1cd831: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 03:11:35.370993 12534 raft_consensus.cc:348] T 642dce5bd0af4e959ca0608b334f9531 P 24da118f679041f7b497d7d91c1cd831 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } }
I20260812 03:11:35.371266 12534 raft_consensus.cc:374] T 642dce5bd0af4e959ca0608b334f9531 P 24da118f679041f7b497d7d91c1cd831 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:11:35.371109 12533 raft_consensus.cc:348] T 642dce5bd0af4e959ca0608b334f9531 P 0f3fa6b5eb1d4f16b072fd0387f60a20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } }
I20260812 03:11:35.371311 12534 raft_consensus.cc:729] T 642dce5bd0af4e959ca0608b334f9531 P 24da118f679041f7b497d7d91c1cd831 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24da118f679041f7b497d7d91c1cd831, State: Initialized, Role: FOLLOWER
I20260812 03:11:35.371393 12533 raft_consensus.cc:374] T 642dce5bd0af4e959ca0608b334f9531 P 0f3fa6b5eb1d4f16b072fd0387f60a20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:11:35.371471 12533 raft_consensus.cc:729] T 642dce5bd0af4e959ca0608b334f9531 P 0f3fa6b5eb1d4f16b072fd0387f60a20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f3fa6b5eb1d4f16b072fd0387f60a20, State: Initialized, Role: FOLLOWER
I20260812 03:11:35.371830 12534 consensus_queue.cc:260] T 642dce5bd0af4e959ca0608b334f9531 P 24da118f679041f7b497d7d91c1cd831 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } }
I20260812 03:11:35.371892 12533 consensus_queue.cc:260] T 642dce5bd0af4e959ca0608b334f9531 P 0f3fa6b5eb1d4f16b072fd0387f60a20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } }
I20260812 03:11:35.372603 12376 heartbeater.cc:503] Master 127.26.250.190:44865 was elected leader, sending a full tablet report...
I20260812 03:11:35.372637 12519 heartbeater.cc:503] Master 127.26.250.190:44865 was elected leader, sending a full tablet report...
I20260812 03:11:35.373010 12533 ts_tablet_manager.cc:1428] T 642dce5bd0af4e959ca0608b334f9531 P 0f3fa6b5eb1d4f16b072fd0387f60a20: Time spent starting tablet: real 0.005s user 0.000s sys 0.006s
I20260812 03:11:35.374056 12534 ts_tablet_manager.cc:1428] T 642dce5bd0af4e959ca0608b334f9531 P 24da118f679041f7b497d7d91c1cd831: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260812 03:11:35.504323 12236 raft_consensus.cc:482] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:11:35.504571 12236 raft_consensus.cc:504] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } }
I20260812 03:11:35.506165 12236 leader_election.cc:288] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f3fa6b5eb1d4f16b072fd0387f60a20 (127.26.250.131:41271), 24da118f679041f7b497d7d91c1cd831 (127.26.250.130:41033)
I20260812 03:11:35.510311 12531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:35.506695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:50679 (local address 127.26.250.130:41033)
0812 03:11:35.506832 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 03:11:35.506837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:11:35.507086 (+ 249us) server_negotiation.cc:407] Connection header received
0812 03:11:35.507251 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:11:35.507255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:11:35.507308 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:11:35.507410 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:11:35.507665 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.508164 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.509283 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.509707 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.509842 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:35.509900 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 03:11:35.510090 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 03:11:35.510165 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:11:35.510370 12540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:35.506543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:41033 (local address 127.26.250.129:50679)
0812 03:11:35.506979 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:35.506992 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:11:35.507124 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:11:35.507420 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:11:35.507423 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:11:35.507441 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:11:35.507549 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:35.507554 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.508286 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.508291 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:35.509142 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:35.509151 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.509843 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.509846 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:35.509946 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:35.509977 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 03:11:35.510071 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 03:11:35.510163 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":5,"thread_start_us":88,"threads_started":1}
I20260812 03:11:35.510516 12530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:35.506589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:44635 (local address 127.26.250.131:41271)
0812 03:11:35.506802 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 03:11:35.506808 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:11:35.506943 (+ 135us) server_negotiation.cc:407] Connection header received
0812 03:11:35.507143 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:11:35.507148 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:11:35.507206 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:11:35.507306 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:11:35.507609 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.508181 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.509362 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.509830 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.509982 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:35.510039 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:11:35.510252 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 03:11:35.510362 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260812 03:11:35.510596 12539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:35.506416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:41271 (local address 127.26.250.129:44635)
0812 03:11:35.506814 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:35.506834 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 03:11:35.507001 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:11:35.507324 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:11:35.507327 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:11:35.507346 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:11:35.507481 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:35.507487 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.508317 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.508322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:35.509211 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:35.509221 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.509977 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.509981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:35.510086 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:35.510117 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 03:11:35.510231 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 03:11:35.510314 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":23,"thread_start_us":102,"threads_started":1}
I20260812 03:11:35.511356 12471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "642dce5bd0af4e959ca0608b334f9531" candidate_uuid: "3754a4ccf1194f39bd37a10b93badcd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" is_pre_election: true
I20260812 03:11:35.511333 12327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "642dce5bd0af4e959ca0608b334f9531" candidate_uuid: "3754a4ccf1194f39bd37a10b93badcd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24da118f679041f7b497d7d91c1cd831" is_pre_election: true
I20260812 03:11:35.511734 12327 raft_consensus.cc:2454] T 642dce5bd0af4e959ca0608b334f9531 P 24da118f679041f7b497d7d91c1cd831 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3754a4ccf1194f39bd37a10b93badcd4 in term 0.
I20260812 03:11:35.511734 12471 raft_consensus.cc:2454] T 642dce5bd0af4e959ca0608b334f9531 P 0f3fa6b5eb1d4f16b072fd0387f60a20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3754a4ccf1194f39bd37a10b93badcd4 in term 0.
I20260812 03:11:35.512320 12111 leader_election.cc:302] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f3fa6b5eb1d4f16b072fd0387f60a20, 3754a4ccf1194f39bd37a10b93badcd4; no voters:
I20260812 03:11:35.512665 12236 raft_consensus.cc:2781] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:11:35.512771 12236 raft_consensus.cc:482] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:11:35.512838 12236 raft_consensus.cc:3037] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:11:35.514026 12236 raft_consensus.cc:504] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } }
I20260812 03:11:35.514459 12236 leader_election.cc:288] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [CANDIDATE]: Term 1 election: Requested vote from peers 0f3fa6b5eb1d4f16b072fd0387f60a20 (127.26.250.131:41271), 24da118f679041f7b497d7d91c1cd831 (127.26.250.130:41033)
I20260812 03:11:35.514901 12471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "642dce5bd0af4e959ca0608b334f9531" candidate_uuid: "3754a4ccf1194f39bd37a10b93badcd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20"
I20260812 03:11:35.514981 12327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "642dce5bd0af4e959ca0608b334f9531" candidate_uuid: "3754a4ccf1194f39bd37a10b93badcd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24da118f679041f7b497d7d91c1cd831"
I20260812 03:11:35.515076 12471 raft_consensus.cc:3037] T 642dce5bd0af4e959ca0608b334f9531 P 0f3fa6b5eb1d4f16b072fd0387f60a20 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:11:35.515161 12327 raft_consensus.cc:3037] T 642dce5bd0af4e959ca0608b334f9531 P 24da118f679041f7b497d7d91c1cd831 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:11:35.516449 12471 raft_consensus.cc:2454] T 642dce5bd0af4e959ca0608b334f9531 P 0f3fa6b5eb1d4f16b072fd0387f60a20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3754a4ccf1194f39bd37a10b93badcd4 in term 1.
I20260812 03:11:35.516495 12327 raft_consensus.cc:2454] T 642dce5bd0af4e959ca0608b334f9531 P 24da118f679041f7b497d7d91c1cd831 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3754a4ccf1194f39bd37a10b93badcd4 in term 1.
I20260812 03:11:35.517341 12113 leader_election.cc:302] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24da118f679041f7b497d7d91c1cd831, 3754a4ccf1194f39bd37a10b93badcd4; no voters:
I20260812 03:11:35.517571 12236 raft_consensus.cc:2781] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:11:35.517809 12236 raft_consensus.cc:686] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 1 LEADER]: Becoming Leader. State: Replica: 3754a4ccf1194f39bd37a10b93badcd4, State: Running, Role: LEADER
I20260812 03:11:35.518117 12236 consensus_queue.cc:237] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } }
I20260812 03:11:35.521728 12024 catalog_manager.cc:5495] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3754a4ccf1194f39bd37a10b93badcd4 (127.26.250.129). New cstate: current_term: 1 leader_uuid: "3754a4ccf1194f39bd37a10b93badcd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } health_report { overall_health: HEALTHY } } }
W20260812 03:11:35.534721 12520 tablet.cc:2369] T 642dce5bd0af4e959ca0608b334f9531 P 0f3fa6b5eb1d4f16b072fd0387f60a20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:11:35.546511 12544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:35.542012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54846 (local address 127.26.250.129:43461)
0812 03:11:35.542406 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0812 03:11:35.542415 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:11:35.542429 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:11:35.542516 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:11:35.542519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:11:35.542577 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:11:35.542717 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:11:35.543036 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.543863 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.544754 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:35.545009 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:35.545103 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:35.545405 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:11:35.545893 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:11:35.546003 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 03:11:35.546064 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 03:11:35.546127 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":139,"threads_started":1}
I20260812 03:11:35.551556 12471 raft_consensus.cc:1261] T 642dce5bd0af4e959ca0608b334f9531 P 0f3fa6b5eb1d4f16b072fd0387f60a20 [term 1 FOLLOWER]: Refusing update from remote peer 3754a4ccf1194f39bd37a10b93badcd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:11:35.551556 12327 raft_consensus.cc:1261] T 642dce5bd0af4e959ca0608b334f9531 P 24da118f679041f7b497d7d91c1cd831 [term 1 FOLLOWER]: Refusing update from remote peer 3754a4ccf1194f39bd37a10b93badcd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:11:35.552459 12541 consensus_queue.cc:1035] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:11:35.552747 12236 consensus_queue.cc:1035] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 03:11:35.558490 12377 tablet.cc:2369] T 642dce5bd0af4e959ca0608b334f9531 P 24da118f679041f7b497d7d91c1cd831: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:11:35.560210 12545 mvcc.cc:205] Tried to move back new op lower bound from 7317521594570764288 to 7317521594445369344. Current Snapshot: MvccSnapshot[applied={T|T < 7317521594570764288}]
I20260812 03:11:35.571319 12024 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:11:35.571560 12024 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:11:35.574527 12024 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a: Sending DeleteTablet for 3 replicas of tablet 642dce5bd0af4e959ca0608b334f9531
I20260812 03:11:35.575382 12159 tablet_service.cc:1514] Processing DeleteTablet for tablet 642dce5bd0af4e959ca0608b334f9531 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:11:35 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:54840
I20260812 03:11:35.575381 12306 tablet_service.cc:1514] Processing DeleteTablet for tablet 642dce5bd0af4e959ca0608b334f9531 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:11:35 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:42632
I20260812 03:11:35.575548 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 12073
I20260812 03:11:35.578953 12554 tablet_replica.cc:330] T 642dce5bd0af4e959ca0608b334f9531 P 24da118f679041f7b497d7d91c1cd831: stopping tablet replica
I20260812 03:11:35.578972 12449 tablet_service.cc:1514] Processing DeleteTablet for tablet 642dce5bd0af4e959ca0608b334f9531 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:11:35 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:36526
I20260812 03:11:35.579452 12554 raft_consensus.cc:2229] T 642dce5bd0af4e959ca0608b334f9531 P 24da118f679041f7b497d7d91c1cd831 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:11:35.579581 12555 tablet_replica.cc:330] T 642dce5bd0af4e959ca0608b334f9531 P 0f3fa6b5eb1d4f16b072fd0387f60a20: stopping tablet replica
I20260812 03:11:35.579905 12554 raft_consensus.cc:2258] T 642dce5bd0af4e959ca0608b334f9531 P 24da118f679041f7b497d7d91c1cd831 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:11:35.579910 12555 raft_consensus.cc:2229] T 642dce5bd0af4e959ca0608b334f9531 P 0f3fa6b5eb1d4f16b072fd0387f60a20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:11:35.580278 12555 raft_consensus.cc:2258] T 642dce5bd0af4e959ca0608b334f9531 P 0f3fa6b5eb1d4f16b072fd0387f60a20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:11:35.581699 12555 ts_tablet_manager.cc:1905] T 642dce5bd0af4e959ca0608b334f9531 P 0f3fa6b5eb1d4f16b072fd0387f60a20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:11:35.581893 12554 ts_tablet_manager.cc:1905] T 642dce5bd0af4e959ca0608b334f9531 P 24da118f679041f7b497d7d91c1cd831: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:11:35.584455 12555 ts_tablet_manager.cc:1918] T 642dce5bd0af4e959ca0608b334f9531 P 0f3fa6b5eb1d4f16b072fd0387f60a20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:11:35.584594 12555 log.cc:1196] T 642dce5bd0af4e959ca0608b334f9531 P 0f3fa6b5eb1d4f16b072fd0387f60a20: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/642dce5bd0af4e959ca0608b334f9531
I20260812 03:11:35.584919 12555 ts_tablet_manager.cc:1939] T 642dce5bd0af4e959ca0608b334f9531 P 0f3fa6b5eb1d4f16b072fd0387f60a20: Deleting consensus metadata
I20260812 03:11:35.585512 12554 ts_tablet_manager.cc:1918] T 642dce5bd0af4e959ca0608b334f9531 P 24da118f679041f7b497d7d91c1cd831: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:11:35.585630 12554 log.cc:1196] T 642dce5bd0af4e959ca0608b334f9531 P 24da118f679041f7b497d7d91c1cd831: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/wal/wals/642dce5bd0af4e959ca0608b334f9531
I20260812 03:11:35.585996 12554 ts_tablet_manager.cc:1939] T 642dce5bd0af4e959ca0608b334f9531 P 24da118f679041f7b497d7d91c1cd831: Deleting consensus metadata
I20260812 03:11:35.586292 12013 catalog_manager.cc:4841] TS 0f3fa6b5eb1d4f16b072fd0387f60a20 (127.26.250.131:41271): tablet 642dce5bd0af4e959ca0608b334f9531 (table test-table [id=6f66120b797d4565be6c11a296d6a2db]) successfully deleted
I20260812 03:11:35.587316 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 12233
I20260812 03:11:35.587288 12015 catalog_manager.cc:4841] TS 24da118f679041f7b497d7d91c1cd831 (127.26.250.130:41033): tablet 642dce5bd0af4e959ca0608b334f9531 (table test-table [id=6f66120b797d4565be6c11a296d6a2db]) successfully deleted
W20260812 03:11:35.587459 12013 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.250.129:43461 (error 108)
I20260812 03:11:35.590853 12529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:35.590419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:43461 (local address 127.0.0.1:54854)
0812 03:11:35.590594 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:35.590704 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.129:43461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 03:11:35.591102 12013 catalog_manager.cc:4584] TS 3754a4ccf1194f39bd37a10b93badcd4 (127.26.250.129:43461): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 642dce5bd0af4e959ca0608b334f9531: Network error: Client connection negotiation failed: client connection to 127.26.250.129:43461: connect: Connection refused (error 111)
I20260812 03:11:35.596273 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 12379
I20260812 03:11:35.605355 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 11995
I20260812 03:11:55.616357 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:44865
--webserver_interface=127.26.250.190
--webserver_port=39479
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:40161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:44865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:11:55.739025 12557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:11:55.739322 12557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:11:55.739375 12557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:11:55.743484 12557 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 03:11:55.743582 12557 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 03:11:55.743613 12557 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 03:11:55.743635 12557 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:11:55.743673 12557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:11:55.743710 12557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:11:55.743728 12557 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:11:55.743749 12557 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:11:55.749305 12557 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:40161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:44865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:44865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=39479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:11:55.750469 12557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:11:55.751482 12557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:11:55.760658 12562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:11:55.760637 12566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:11:55.760625 12563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:11:55.760691 12557 server_base.cc:1034] running on GCE node
I20260812 03:11:55.761456 12557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:11:55.762686 12557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:11:55.763942 12557 hybrid_clock.cc:648] HybridClock initialized: now 1786504315763902 us; error 57 us; skew 500 ppm
Aug 12 03:11:55 dist-test-slave-zpfg krb5kdc[11413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504315, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:11:55.774142 12557 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:11:55.775396 12557 webserver.cc:460] Webserver started at http://127.26.250.190:39479/ using document root <none> and password file <none>
I20260812 03:11:55.776098 12557 fs_manager.cc:359] Metadata directory not provided
I20260812 03:11:55.776223 12557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:11:55.780779 12557 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:11:55.783356 12574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:11:55.784919 12557 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:11:55.785099 12557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "8f7ac59b76224979a06891a373d00f2a"
format_stamp: "Formatted at 2026-08-12 03:11:13 on dist-test-slave-zpfg"
server_key: "4c1016cd34d3c107df325752d87c2211"
server_key_iv: "2e8a470e634bbb3118c8cc1e88db6735"
server_key_version: "encryptionkey@0"
I20260812 03:11:55.785742 12557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:11:55.811427 12557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:11:55.815476 12557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:11:55.815673 12557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:11:55.824385 12557 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:44865
I20260812 03:11:55.824699 12630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:44865 every 8 connection(s)
I20260812 03:11:55.825716 12557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:11:55.830539 12631 sys_catalog.cc:263] Verifying existing consensus state
I20260812 03:11:55.832315 12631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a: Bootstrap starting.
I20260812 03:11:55.835366 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 12557
I20260812 03:11:55.835925 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:43461
--local_ip_for_outbound_sockets=127.26.250.129
--tserver_master_addrs=127.26.250.190:44865
--webserver_port=45551
--webserver_interface=127.26.250.129
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:40161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:11:55.852964 12631 log.cc:824] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:11:55.865481 12631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:11:55.865927 12631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a: Bootstrap complete.
I20260812 03:11:55.870246 12631 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7ac59b76224979a06891a373d00f2a" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 44865 } }
I20260812 03:11:55.870819 12631 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f7ac59b76224979a06891a373d00f2a, State: Initialized, Role: FOLLOWER
I20260812 03:11:55.871349 12631 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7ac59b76224979a06891a373d00f2a" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 44865 } }
I20260812 03:11:55.871537 12631 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:11:55.871614 12631 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:11:55.871732 12631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [term 2 FOLLOWER]: Advancing to term 3
I20260812 03:11:55.873832 12631 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7ac59b76224979a06891a373d00f2a" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 44865 } }
I20260812 03:11:55.874210 12631 leader_election.cc:302] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f7ac59b76224979a06891a373d00f2a; no voters:
I20260812 03:11:55.874648 12631 leader_election.cc:288] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 03:11:55.874763 12635 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [term 3 FOLLOWER]: Leader election won for term 3
I20260812 03:11:55.875036 12635 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [term 3 LEADER]: Becoming Leader. State: Replica: 8f7ac59b76224979a06891a373d00f2a, State: Running, Role: LEADER
I20260812 03:11:55.875430 12635 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7ac59b76224979a06891a373d00f2a" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 44865 } }
I20260812 03:11:55.876029 12631 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:11:55.876922 12636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8f7ac59b76224979a06891a373d00f2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7ac59b76224979a06891a373d00f2a" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 44865 } } }
I20260812 03:11:55.877058 12636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [sys.catalog]: This master's current role is: LEADER
I20260812 03:11:55.877489 12637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f7ac59b76224979a06891a373d00f2a. Latest consensus state: current_term: 3 leader_uuid: "8f7ac59b76224979a06891a373d00f2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7ac59b76224979a06891a373d00f2a" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 44865 } } }
I20260812 03:11:55.877637 12637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a [sys.catalog]: This master's current role is: LEADER
I20260812 03:11:55.881402 12652 catalog_manager.cc:1259] Loaded cluster ID: c4438346609941f4befa8512f1269eb2
I20260812 03:11:55.881479 12652 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a: loading cluster ID for follower catalog manager: success
I20260812 03:11:55.884450 12652 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8f7ac59b76224979a06891a373d00f2a: acquiring CA information for follower catalog manager: success
F20260812 03:11:55.886008 12652 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0a544a59ad google::LogMessage::Fail() at ??:0
@ 0x7f0a544a888a google::LogMessage::SendToLog() at ??:0
@ 0x7f0a544a54de google::LogMessage::Flush() at ??:0
@ 0x7f0a544a6a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f0a5716918d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f0a5ee0a989 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7f0a5ede7023 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7f0a5ede754b kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7f0a5ede0652 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7f0a5eddf9e9 _ZZN4kudu6master21CatalogManagerBgTasks4InitEvENKUlvE_clEv at ??:0
@ 0x7f0a5ee26541 _ZNSt17_Function_handlerIFvvEZN4kudu6master21CatalogManagerBgTasks4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f0a566e8512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f0a57d2c6db start_thread at ??:0
@ 0x7f0a52fa361f clone at ??:0
W20260812 03:11:55.988415 12633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:11:55.988703 12633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:11:55.988749 12633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:11:55.993747 12633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:11:55.993846 12633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:11:55.993955 12633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:11:56.000587 12633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:40161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:43461
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=45551
--tserver_master_addrs=127.26.250.190:44865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:11:56.001875 12633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:11:56.003082 12633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:11:56.014946 12672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:11:56.015043 12673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:11:56.015249 12675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:11:56.015614 12633 server_base.cc:1034] running on GCE node
I20260812 03:11:56.016222 12633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:11:56.017145 12633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:11:56.018406 12633 hybrid_clock.cc:648] HybridClock initialized: now 1786504316018368 us; error 36 us; skew 500 ppm
Aug 12 03:11:56 dist-test-slave-zpfg krb5kdc[11413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504316, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:11:56.022544 12633 init.cc:377] Logged in from keytab as kudu/127.26.250.129@KRBTEST.COM (short username kudu)
I20260812 03:11:56.023780 12633 webserver.cc:460] Webserver started at http://127.26.250.129:45551/ using document root <none> and password file <none>
I20260812 03:11:56.024477 12633 fs_manager.cc:359] Metadata directory not provided
I20260812 03:11:56.024605 12633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:11:56.029498 12633 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 03:11:56.032488 12682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:11:56.034011 12633 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 03:11:56.034190 12633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "3754a4ccf1194f39bd37a10b93badcd4"
format_stamp: "Formatted at 2026-08-12 03:11:13 on dist-test-slave-zpfg"
server_key: "3aa1b0adc55e07c53302d5775c1836ea"
server_key_iv: "981d9ad35af1a2cf103e287ac27a0b85"
server_key_version: "encryptionkey@0"
I20260812 03:11:56.034987 12633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:11:56.046520 12633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:11:56.049974 12633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:11:56.050192 12633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:11:56.050909 12633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:11:56.052100 12691 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 03:11:56.057940 12633 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 03:11:56.058071 12633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260812 03:11:56.058182 12633 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 03:11:56.060282 12633 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 03:11:56.060370 12633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260812 03:11:56.060671 12691 tablet_bootstrap.cc:492] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4: Bootstrap starting.
I20260812 03:11:56.073380 12633 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:43461
I20260812 03:11:56.073944 12807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:43461 every 8 connection(s)
I20260812 03:11:56.074707 12633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:11:56.075121 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 12633
I20260812 03:11:56.075655 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:41033
--local_ip_for_outbound_sockets=127.26.250.130
--tserver_master_addrs=127.26.250.190:44865
--webserver_port=35029
--webserver_interface=127.26.250.130
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:40161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:11:56.088578 12691 log.cc:824] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4: Log is configured to *not* fsync() on all Append() calls
I20260812 03:11:56.099792 12691 tablet_bootstrap.cc:492] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 03:11:56.100329 12691 tablet_bootstrap.cc:492] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4: Bootstrap complete.
I20260812 03:11:56.102044 12691 ts_tablet_manager.cc:1397] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4: Time spent bootstrapping tablet: real 0.042s user 0.026s sys 0.009s
I20260812 03:11:56.105692 12691 raft_consensus.cc:348] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } }
I20260812 03:11:56.106331 12691 raft_consensus.cc:729] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3754a4ccf1194f39bd37a10b93badcd4, State: Initialized, Role: FOLLOWER
I20260812 03:11:56.106923 12691 consensus_queue.cc:260] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } }
I20260812 03:11:56.108098 12691 ts_tablet_manager.cc:1428] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4: Time spent starting tablet: real 0.006s user 0.009s sys 0.000s
W20260812 03:11:56.122629 12811 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:11:56.077785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.129:45237)
0812 03:11:56.078477 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:56.078546 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 03:11:56.079751 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:11:56.121592 (+ 41841us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":184,"threads_started":1}
W20260812 03:11:56.123221 12808 heartbeater.cc:644] Failed to heartbeat to 127.26.250.190:44865 (0 consecutive failures): Network error: Failed to ping master at 127.26.250.190:44865: Client connection negotiation failed: client connection to 127.26.250.190:44865: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 03:11:56.124100 12811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:56.123757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.129:56137)
0812 03:11:56.123928 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:56.124023 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 03:11:56.124930 12811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:56.124689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.129:46693)
0812 03:11:56.124810 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:56.124866 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 03:11:56.125133 12808 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:11:56.211957 12810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:11:56.212297 12810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:11:56.212388 12810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:11:56.217491 12810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:11:56.217581 12810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:11:56.217679 12810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:11:56.224125 12810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:40161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:41033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=35029
--tserver_master_addrs=127.26.250.190:44865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:11:56.225311 12810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:11:56.226490 12810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:11:56.238102 12820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:11:56.238094 12819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:11:56.238094 12822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:11:56.238746 12810 server_base.cc:1034] running on GCE node
I20260812 03:11:56.239238 12810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:11:56.240036 12810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:11:56.241328 12810 hybrid_clock.cc:648] HybridClock initialized: now 1786504316241285 us; error 60 us; skew 500 ppm
Aug 12 03:11:56 dist-test-slave-zpfg krb5kdc[11413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504316, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:11:56.244791 12810 init.cc:377] Logged in from keytab as kudu/127.26.250.130@KRBTEST.COM (short username kudu)
I20260812 03:11:56.245852 12810 webserver.cc:460] Webserver started at http://127.26.250.130:35029/ using document root <none> and password file <none>
I20260812 03:11:56.246785 12810 fs_manager.cc:359] Metadata directory not provided
I20260812 03:11:56.246868 12810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:11:56.251461 12810 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260812 03:11:56.253796 12829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:11:56.255484 12810 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:11:56.255601 12810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "24da118f679041f7b497d7d91c1cd831"
format_stamp: "Formatted at 2026-08-12 03:11:13 on dist-test-slave-zpfg"
server_key: "f3fb148c85edec6f70e7a0b0dd9f49e3"
server_key_iv: "11ce5d20e7f36ce904362b0fd6b3bde4"
server_key_version: "encryptionkey@0"
I20260812 03:11:56.256217 12810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:11:56.271080 12810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:11:56.274816 12810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:11:56.274996 12810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:11:56.275607 12810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:11:56.276466 12810 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:11:56.276518 12810 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:11:56.276559 12810 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:11:56.276575 12810 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:11:56.287153 12810 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:41033
I20260812 03:11:56.287288 12947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:41033 every 8 connection(s)
I20260812 03:11:56.288300 12810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:11:56.289194 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 12810
I20260812 03:11:56.289601 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:41271
--local_ip_for_outbound_sockets=127.26.250.131
--tserver_master_addrs=127.26.250.190:44865
--webserver_port=41767
--webserver_interface=127.26.250.131
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:40161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:11:56.292819 12951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:56.291001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.130:47261)
0812 03:11:56.291880 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:56.292038 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":152,"threads_started":1}
W20260812 03:11:56.293462 12948 heartbeater.cc:644] Failed to heartbeat to 127.26.250.190:44865 (0 consecutive failures): Network error: Failed to ping master at 127.26.250.190:44865: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
I20260812 03:11:56.294189 12951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:56.293939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.130:59515)
0812 03:11:56.294062 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:56.294127 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 03:11:56.295094 12951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:56.294774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.130:60363)
0812 03:11:56.294924 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:56.294993 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 03:11:56.295310 12948 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 03:11:56.410355 12950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:11:56.410727 12950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:11:56.410797 12950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:11:56.414976 12950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:11:56.415041 12950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:11:56.415104 12950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:11:56.420729 12950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:40161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:41271
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=41767
--tserver_master_addrs=127.26.250.190:44865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:11:56.421849 12950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:11:56.422953 12950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:11:56.433633 12959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:11:56.433657 12957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:11:56.433625 12956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:11:56.434110 12950 server_base.cc:1034] running on GCE node
I20260812 03:11:56.434619 12950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:11:56.435362 12950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:11:56.436561 12950 hybrid_clock.cc:648] HybridClock initialized: now 1786504316436512 us; error 64 us; skew 500 ppm
Aug 12 03:11:56 dist-test-slave-zpfg krb5kdc[11413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504316, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:11:56.439689 12950 init.cc:377] Logged in from keytab as kudu/127.26.250.131@KRBTEST.COM (short username kudu)
I20260812 03:11:56.440747 12950 webserver.cc:460] Webserver started at http://127.26.250.131:41767/ using document root <none> and password file <none>
I20260812 03:11:56.441366 12950 fs_manager.cc:359] Metadata directory not provided
I20260812 03:11:56.441463 12950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:11:56.445976 12950 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:11:56.448552 12966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:11:56.450083 12950 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 03:11:56.450207 12950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20"
format_stamp: "Formatted at 2026-08-12 03:11:14 on dist-test-slave-zpfg"
server_key: "77fce3b673be7368add2a0bb4ee68587"
server_key_iv: "601dc1cf6460930ee9aed83c38df19c2"
server_key_version: "encryptionkey@0"
I20260812 03:11:56.450852 12950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:11:56.467483 12950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:11:56.471110 12950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:11:56.471298 12950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:11:56.471885 12950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:11:56.472735 12950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:11:56.472788 12950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:11:56.472841 12950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:11:56.472863 12950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:11:56.482723 12950 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:41271
I20260812 03:11:56.482805 13089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:41271 every 8 connection(s)
I20260812 03:11:56.483872 12950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:11:56.488113 13092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:56.486290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.131:47279)
0812 03:11:56.487162 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:56.487309 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":145,"threads_started":1}
I20260812 03:11:56.488565 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 12950
W20260812 03:11:56.488610 13090 heartbeater.cc:644] Failed to heartbeat to 127.26.250.190:44865 (0 consecutive failures): Network error: Failed to ping master at 127.26.250.190:44865: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
I20260812 03:11:56.489394 13092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:56.489149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.131:52053)
0812 03:11:56.489277 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:56.489335 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 03:11:56.490026 13092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:56.489840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.131:38687)
0812 03:11:56.489934 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:56.489978 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 03:11:56.490239 13090 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 03:11:57.126614 13094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:57.126039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.129:47157)
0812 03:11:57.126396 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:57.126460 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":100,"threads_started":1}
I20260812 03:11:57.296429 13095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:57.295946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.130:40337)
0812 03:11:57.296242 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:57.296306 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":84,"threads_started":1}
I20260812 03:11:57.491375 13096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:57.490910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.131:54679)
0812 03:11:57.491185 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:57.491251 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":80,"threads_started":1}
I20260812 03:11:57.666548 13097 raft_consensus.cc:482] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:11:57.666819 13097 raft_consensus.cc:504] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } }
I20260812 03:11:57.668054 13097 leader_election.cc:288] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0f3fa6b5eb1d4f16b072fd0387f60a20 (127.26.250.131:41271), 24da118f679041f7b497d7d91c1cd831 (127.26.250.130:41033)
Aug 12 03:11:57 dist-test-slave-zpfg krb5kdc[11413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504316, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for kudu/127.26.250.131@KRBTEST.COM
Aug 12 03:11:57 dist-test-slave-zpfg krb5kdc[11413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504316, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for kudu/127.26.250.130@KRBTEST.COM
I20260812 03:11:57.688822 13102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:57.668679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:38131 (local address 127.26.250.130:41033)
0812 03:11:57.668978 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0812 03:11:57.669011 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0812 03:11:57.669058 (+ 47us) server_negotiation.cc:407] Connection header received
0812 03:11:57.669999 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:11:57.670009 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:11:57.670542 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:11:57.670916 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:11:57.672888 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:57.674701 (+ 1813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:57.676898 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:57.677688 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:57.677890 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:57.682228 (+ 4338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:11:57.682237 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:11:57.682251 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:11:57.682295 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:11:57.685107 (+ 2812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:11:57.685737 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:11:57.685743 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:11:57.685747 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:11:57.685912 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:11:57.686532 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:11:57.686536 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:11:57.686538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:11:57.687160 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:11:57.687302 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 03:11:57.688265 (+ 963us) server_negotiation.cc:299] Negotiation successful
0812 03:11:57.688609 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":94,"threads_started":1}
I20260812 03:11:57.688810 13098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:57.668289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:41271 (local address 127.26.250.129:35121)
0812 03:11:57.668707 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:57.668727 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 03:11:57.668857 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:11:57.671062 (+ 2205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:11:57.671069 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:11:57.671727 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:11:57.672635 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:57.672661 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:57.674902 (+ 2241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:57.674906 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:57.676757 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:57.676764 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:57.677880 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:57.677884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:57.678291 (+ 407us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:57.679215 (+ 924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:11:57.679243 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:11:57.681725 (+ 2482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:11:57.685018 (+ 3293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:11:57.685024 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:11:57.685039 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:11:57.685533 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:11:57.686047 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:11:57.686050 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:11:57.686052 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:11:57.686414 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:11:57.687346 (+ 932us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:11:57.687354 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:11:57.687817 (+ 463us) client_negotiation.cc:769] Sending connection context
0812 03:11:57.688101 (+ 284us) client_negotiation.cc:240] Negotiation successful
0812 03:11:57.688618 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":674,"spinlock_wait_cycles":31232,"thread_start_us":87,"threads_started":1}
I20260812 03:11:57.688843 13101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:57.668499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:35121 (local address 127.26.250.131:41271)
0812 03:11:57.669081 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0812 03:11:57.669102 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 03:11:57.669135 (+ 33us) server_negotiation.cc:407] Connection header received
0812 03:11:57.669999 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:11:57.670009 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:11:57.670542 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:11:57.670916 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:11:57.673051 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:57.674680 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:57.676898 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:11:57.677688 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:11:57.677910 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:57.682068 (+ 4158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:11:57.682076 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:11:57.682097 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:11:57.682145 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:11:57.684832 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:11:57.685714 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:11:57.685722 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:11:57.685728 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:11:57.685922 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:11:57.686532 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:11:57.686536 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:11:57.686538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:11:57.687160 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:11:57.687302 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 03:11:57.688265 (+ 963us) server_negotiation.cc:299] Negotiation successful
0812 03:11:57.688609 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":112,"threads_started":1}
I20260812 03:11:57.688812 13100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:57.668442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:41033 (local address 127.26.250.129:38131)
0812 03:11:57.668809 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:57.668823 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:11:57.668912 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:11:57.671062 (+ 2150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:11:57.671069 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:11:57.671727 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:11:57.672635 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:57.672661 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:57.674905 (+ 2244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:57.674907 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:57.676721 (+ 1814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:11:57.676739 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:11:57.677880 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:11:57.677884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:11:57.678297 (+ 413us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:11:57.679335 (+ 1038us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:11:57.679354 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:11:57.681969 (+ 2615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:11:57.685300 (+ 3331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:11:57.685303 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:11:57.685305 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:11:57.685617 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:11:57.686047 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:11:57.686050 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:11:57.686063 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:11:57.686414 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:11:57.687346 (+ 932us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:11:57.687354 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:11:57.687817 (+ 463us) client_negotiation.cc:769] Sending connection context
0812 03:11:57.688101 (+ 284us) client_negotiation.cc:240] Negotiation successful
0812 03:11:57.688556 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":457,"spinlock_wait_cycles":38528,"thread_start_us":88,"threads_started":1}
W20260812 03:11:57.691817 12688 leader_election.cc:341] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 24da118f679041f7b497d7d91c1cd831 (127.26.250.130:41033): Not found: Tablet not found: 642dce5bd0af4e959ca0608b334f9531
W20260812 03:11:57.691992 12685 leader_election.cc:341] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0f3fa6b5eb1d4f16b072fd0387f60a20 (127.26.250.131:41271): Not found: Tablet not found: 642dce5bd0af4e959ca0608b334f9531
I20260812 03:11:57.692104 12685 leader_election.cc:302] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3754a4ccf1194f39bd37a10b93badcd4; no voters: 0f3fa6b5eb1d4f16b072fd0387f60a20, 24da118f679041f7b497d7d91c1cd831
I20260812 03:11:57.692431 13097 raft_consensus.cc:2726] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:11:58.127918 13098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:58.127539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.129:37071)
0812 03:11:58.127744 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:58.127823 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260812 03:11:58.297963 13103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:58.297403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.130:58803)
0812 03:11:58.297757 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:58.297838 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":92,"threads_started":1}
I20260812 03:11:58.492623 13104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:58.492166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.131:48567)
0812 03:11:58.492456 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:58.492525 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":83,"threads_started":1}
I20260812 03:11:59.129674 13105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:59.129053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.129:57791)
0812 03:11:59.129485 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:59.129561 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":90,"threads_started":1}
I20260812 03:11:59.299625 13106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:59.299038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.130:46843)
0812 03:11:59.299443 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:59.299507 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":101,"threads_started":1}
I20260812 03:11:59.494154 13107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:11:59.493628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.131:47327)
0812 03:11:59.493951 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0812 03:11:59.494019 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":87,"threads_started":1}
I20260812 03:11:59.923225 13108 raft_consensus.cc:482] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:11:59.923391 13108 raft_consensus.cc:504] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } }
I20260812 03:11:59.923969 13108 leader_election.cc:288] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0f3fa6b5eb1d4f16b072fd0387f60a20 (127.26.250.131:41271), 24da118f679041f7b497d7d91c1cd831 (127.26.250.130:41033)
W20260812 03:11:59.925014 12685 leader_election.cc:341] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0f3fa6b5eb1d4f16b072fd0387f60a20 (127.26.250.131:41271): Not found: Tablet not found: 642dce5bd0af4e959ca0608b334f9531
W20260812 03:11:59.925132 12688 leader_election.cc:341] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 24da118f679041f7b497d7d91c1cd831 (127.26.250.130:41033): Not found: Tablet not found: 642dce5bd0af4e959ca0608b334f9531
I20260812 03:11:59.925273 12688 leader_election.cc:302] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3754a4ccf1194f39bd37a10b93badcd4; no voters: 0f3fa6b5eb1d4f16b072fd0387f60a20, 24da118f679041f7b497d7d91c1cd831
I20260812 03:11:59.925472 13108 raft_consensus.cc:2726] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:12:00.131433 13109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:00.130775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.129:42845)
0812 03:12:00.131241 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:00.131324 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":122,"threads_started":1}
I20260812 03:12:00.301079 13110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:00.300579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.130:53937)
0812 03:12:00.300907 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:00.300976 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":120,"threads_started":1}
I20260812 03:12:00.495630 13112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:00.495171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.131:44599)
0812 03:12:00.495469 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:00.495530 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":99,"threads_started":1}
I20260812 03:12:01.132918 13113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:01.132411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.129:48789)
0812 03:12:01.132729 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:01.132790 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":95,"threads_started":1}
I20260812 03:12:01.302539 13114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:01.301991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.130:58083)
0812 03:12:01.302315 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:01.302423 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":88,"threads_started":1}
I20260812 03:12:01.497175 13115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:01.496659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.131:50655)
0812 03:12:01.496998 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:01.497072 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":93,"threads_started":1}
I20260812 03:12:02.134478 13116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:02.133895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.129:46145)
0812 03:12:02.134219 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:02.134320 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":142,"threads_started":1}
I20260812 03:12:02.145123 13117 raft_consensus.cc:482] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:12:02.145305 13117 raft_consensus.cc:504] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } }
I20260812 03:12:02.145891 13117 leader_election.cc:288] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0f3fa6b5eb1d4f16b072fd0387f60a20 (127.26.250.131:41271), 24da118f679041f7b497d7d91c1cd831 (127.26.250.130:41033)
W20260812 03:12:02.146764 12685 leader_election.cc:341] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0f3fa6b5eb1d4f16b072fd0387f60a20 (127.26.250.131:41271): Not found: Tablet not found: 642dce5bd0af4e959ca0608b334f9531
W20260812 03:12:02.146917 12688 leader_election.cc:341] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 24da118f679041f7b497d7d91c1cd831 (127.26.250.130:41033): Not found: Tablet not found: 642dce5bd0af4e959ca0608b334f9531
I20260812 03:12:02.147049 12688 leader_election.cc:302] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3754a4ccf1194f39bd37a10b93badcd4; no voters: 0f3fa6b5eb1d4f16b072fd0387f60a20, 24da118f679041f7b497d7d91c1cd831
I20260812 03:12:02.147300 13117 raft_consensus.cc:2726] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:12:02.303872 13118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:02.303408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.130:58163)
0812 03:12:02.303701 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:02.303763 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":88,"threads_started":1}
I20260812 03:12:02.498870 13119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:02.498123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.131:43571)
0812 03:12:02.498650 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:02.498715 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":290,"threads_started":1}
I20260812 03:12:03.136106 13120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:03.135582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.129:40295)
0812 03:12:03.135931 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:03.136004 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":87,"threads_started":1}
I20260812 03:12:03.305433 13122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:03.304772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.130:56145)
0812 03:12:03.305209 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:03.305307 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":91,"threads_started":1}
I20260812 03:12:03.500550 13123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:03.499953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.131:55901)
0812 03:12:03.500376 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:03.500445 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":97,"threads_started":1}
I20260812 03:12:04.137598 13124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:04.137071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.129:35223)
0812 03:12:04.137426 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:04.137495 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":135,"threads_started":1}
I20260812 03:12:04.307132 13125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:04.306643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.130:58257)
0812 03:12:04.306952 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:04.307023 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":80,"threads_started":1}
I20260812 03:12:04.502092 13126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:04.501600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.131:40889)
0812 03:12:04.501918 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:04.501982 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":87,"threads_started":1}
I20260812 03:12:05.139369 13127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:05.138716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.129:38807)
0812 03:12:05.139142 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:05.139232 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":122,"threads_started":1}
I20260812 03:12:05.308710 13128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:05.308046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.130:58425)
0812 03:12:05.308530 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:05.308595 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":207,"threads_started":1}
I20260812 03:12:05.503598 13129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:05.503128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.131:52325)
0812 03:12:05.503425 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:05.503491 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":86,"threads_started":1}
I20260812 03:12:06.109026 13130 raft_consensus.cc:482] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:12:06.109230 13130 raft_consensus.cc:504] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } }
I20260812 03:12:06.109745 13130 leader_election.cc:288] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0f3fa6b5eb1d4f16b072fd0387f60a20 (127.26.250.131:41271), 24da118f679041f7b497d7d91c1cd831 (127.26.250.130:41033)
W20260812 03:12:06.110680 12688 leader_election.cc:341] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 24da118f679041f7b497d7d91c1cd831 (127.26.250.130:41033): Not found: Tablet not found: 642dce5bd0af4e959ca0608b334f9531
W20260812 03:12:06.110778 12685 leader_election.cc:341] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0f3fa6b5eb1d4f16b072fd0387f60a20 (127.26.250.131:41271): Not found: Tablet not found: 642dce5bd0af4e959ca0608b334f9531
I20260812 03:12:06.110901 12685 leader_election.cc:302] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3754a4ccf1194f39bd37a10b93badcd4; no voters: 0f3fa6b5eb1d4f16b072fd0387f60a20, 24da118f679041f7b497d7d91c1cd831
I20260812 03:12:06.111135 13130 raft_consensus.cc:2726] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:12:06.141119 13131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:06.140650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.129:41953)
0812 03:12:06.140955 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:06.141021 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":86,"threads_started":1}
I20260812 03:12:06.310427 13133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:06.309851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.130:38145)
0812 03:12:06.310182 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:06.310253 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":110,"threads_started":1}
I20260812 03:12:06.505119 13134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:06.504539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.131:39729)
0812 03:12:06.504867 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:06.504934 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":100,"threads_started":1}
I20260812 03:12:07.142639 13135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:07.142064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.129:45825)
0812 03:12:07.142468 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:07.142538 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":92,"threads_started":1}
I20260812 03:12:07.312029 13136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:07.311490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.130:48363)
0812 03:12:07.311847 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:07.311922 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":87,"threads_started":1}
I20260812 03:12:07.506734 13137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:07.506020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.131:55495)
0812 03:12:07.506558 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:07.506630 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":243,"threads_started":1}
I20260812 03:12:07.738595 13139 raft_consensus.cc:482] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:12:07.738739 13139 raft_consensus.cc:504] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f3fa6b5eb1d4f16b072fd0387f60a20" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 41271 } } peers { permanent_uuid: "24da118f679041f7b497d7d91c1cd831" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41033 } } peers { permanent_uuid: "3754a4ccf1194f39bd37a10b93badcd4" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 43461 } }
I20260812 03:12:07.739333 13139 leader_election.cc:288] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0f3fa6b5eb1d4f16b072fd0387f60a20 (127.26.250.131:41271), 24da118f679041f7b497d7d91c1cd831 (127.26.250.130:41033)
W20260812 03:12:07.740206 12685 leader_election.cc:341] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0f3fa6b5eb1d4f16b072fd0387f60a20 (127.26.250.131:41271): Not found: Tablet not found: 642dce5bd0af4e959ca0608b334f9531
W20260812 03:12:07.740337 12688 leader_election.cc:341] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 24da118f679041f7b497d7d91c1cd831 (127.26.250.130:41033): Not found: Tablet not found: 642dce5bd0af4e959ca0608b334f9531
I20260812 03:12:07.740443 12688 leader_election.cc:302] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3754a4ccf1194f39bd37a10b93badcd4; no voters: 0f3fa6b5eb1d4f16b072fd0387f60a20, 24da118f679041f7b497d7d91c1cd831
I20260812 03:12:07.740613 13139 raft_consensus.cc:2726] T 642dce5bd0af4e959ca0608b334f9531 P 3754a4ccf1194f39bd37a10b93badcd4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 03:12:08.143891 13140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:08.143441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.129:44861)
0812 03:12:08.143718 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:08.143781 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":78,"threads_started":1}
I20260812 03:12:08.313572 13141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:08.312940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.130:47447)
0812 03:12:08.313371 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:08.313458 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":83,"threads_started":1}
I20260812 03:12:08.508173 13142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:08.507577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.131:54245)
0812 03:12:08.507867 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:08.507934 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":85,"threads_started":1}
I20260812 03:12:09.145264 13143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:09.144791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.129:41151)
0812 03:12:09.145093 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:09.145161 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":83,"threads_started":1}
I20260812 03:12:09.315388 13144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:09.314788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.130:46923)
0812 03:12:09.315135 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:09.315208 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":120,"threads_started":1}
I20260812 03:12:09.509501 13145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:09.509054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.131:39363)
0812 03:12:09.509343 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:09.509402 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":98,"threads_started":1}
I20260812 03:12:10.146579 13146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:10.146087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.129:45201)
0812 03:12:10.146410 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:10.146476 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":104,"threads_started":1}
I20260812 03:12:10.316941 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:10.316459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.130:40103)
0812 03:12:10.316766 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:10.316834 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":95,"threads_started":1}
I20260812 03:12:10.510989 13148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:10.510484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.131:38135)
0812 03:12:10.510788 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:10.510886 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":89,"threads_started":1}
I20260812 03:12:11.148099 13149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:11.147677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.129:35389)
0812 03:12:11.147934 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:11.147988 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":81,"threads_started":1}
I20260812 03:12:11.318811 13150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:11.318001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:44865 (local address 127.26.250.130:47765)
0812 03:12:11.318309 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:11.318705 (+ 396us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.190:44865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":92,"threads_started":1}
I20260812 03:12:11.494743 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 12633
I20260812 03:12:11.503614 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 12810
I20260812 03:12:11.511224 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 12950
2026-08-12T03:12:11Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59597 ms)
[----------] 30 tests from SecurityITest (256227 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:12:11 dist-test-slave-zpfg krb5kdc[13153](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:12:11 dist-test-slave-zpfg krb5kdc[13153](info): set up 2 sockets
Aug 12 03:12:11 dist-test-slave-zpfg krb5kdc[13153](info): commencing operation
krb5kdc: starting...
W20260812 03:12:12.651324 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:12:12 dist-test-slave-zpfg krb5kdc[13153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:12:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:12:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:12:12.798198 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:36407
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:46073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:36407
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:12.920619 13167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:12.920980 13167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:12.921065 13167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:12.925448 13167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:12.925559 13167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:12.925601 13167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:12.925648 13167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:12.925673 13167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:12.931355 13167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:46073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:36407
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:36407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:12.932605 13167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:12.933770 13167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:12.944417 13172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:12.944501 13167 server_base.cc:1034] running on GCE node
W20260812 03:12:12.944428 13173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:12.944439 13175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:12.945356 13167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:12.946836 13167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:12.948060 13167 hybrid_clock.cc:648] HybridClock initialized: now 1786504332948039 us; error 34 us; skew 500 ppm
Aug 12 03:12:12 dist-test-slave-zpfg krb5kdc[13153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504332, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:12.951241 13167 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:12:12.952299 13167 webserver.cc:460] Webserver started at http://127.26.250.190:43247/ using document root <none> and password file <none>
I20260812 03:12:12.952932 13167 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:12.953037 13167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:12.953284 13167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:12.955596 13167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "0de9b9e1e08d477391490d612b574a33"
format_stamp: "Formatted at 2026-08-12 03:12:12 on dist-test-slave-zpfg"
server_key: "c8acfa5582ac8b70a48d03886346745b"
server_key_iv: "0e9dd7bd905d3ffb6a7aa500f420b08d"
server_key_version: "encryptionkey@0"
I20260812 03:12:12.956248 13167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "0de9b9e1e08d477391490d612b574a33"
format_stamp: "Formatted at 2026-08-12 03:12:12 on dist-test-slave-zpfg"
server_key: "c8acfa5582ac8b70a48d03886346745b"
server_key_iv: "0e9dd7bd905d3ffb6a7aa500f420b08d"
server_key_version: "encryptionkey@0"
I20260812 03:12:12.960453 13167 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:12:12.963408 13182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:12.964775 13167 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:12:12.964931 13167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "0de9b9e1e08d477391490d612b574a33"
format_stamp: "Formatted at 2026-08-12 03:12:12 on dist-test-slave-zpfg"
server_key: "c8acfa5582ac8b70a48d03886346745b"
server_key_iv: "0e9dd7bd905d3ffb6a7aa500f420b08d"
server_key_version: "encryptionkey@0"
I20260812 03:12:12.965061 13167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:12.976481 13167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:12.979918 13167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:12.980139 13167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:12.987815 13167 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:36407
I20260812 03:12:12.987969 13238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:36407 every 8 connection(s)
I20260812 03:12:12.988874 13167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:12:12.992000 13239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:12.997074 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 13167
I20260812 03:12:12.997215 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:12:12.997525 27626 external_mini_cluster.cc:1428] Setting key e286d07fa886a15a8ea729a2496c5e71
I20260812 03:12:12.998174 13239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0de9b9e1e08d477391490d612b574a33: Bootstrap starting.
I20260812 03:12:13.000861 13239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0de9b9e1e08d477391490d612b574a33: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:13.001682 13239 log.cc:824] T 00000000000000000000000000000000 P 0de9b9e1e08d477391490d612b574a33: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:13.003928 13239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0de9b9e1e08d477391490d612b574a33: No bootstrap required, opened a new log
Aug 12 03:12:13 dist-test-slave-zpfg krb5kdc[13153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:12:13.008047 13239 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0de9b9e1e08d477391490d612b574a33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0de9b9e1e08d477391490d612b574a33" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 36407 } }
I20260812 03:12:13.008405 13239 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0de9b9e1e08d477391490d612b574a33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:13.008493 13239 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0de9b9e1e08d477391490d612b574a33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0de9b9e1e08d477391490d612b574a33, State: Initialized, Role: FOLLOWER
I20260812 03:12:13.009078 13239 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0de9b9e1e08d477391490d612b574a33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0de9b9e1e08d477391490d612b574a33" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 36407 } }
I20260812 03:12:13.009260 13239 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0de9b9e1e08d477391490d612b574a33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:12:13.009351 13239 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0de9b9e1e08d477391490d612b574a33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:12:13.009491 13239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0de9b9e1e08d477391490d612b574a33 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:13.010603 13239 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0de9b9e1e08d477391490d612b574a33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0de9b9e1e08d477391490d612b574a33" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 36407 } }
I20260812 03:12:13.011055 13239 leader_election.cc:302] T 00000000000000000000000000000000 P 0de9b9e1e08d477391490d612b574a33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0de9b9e1e08d477391490d612b574a33; no voters:
I20260812 03:12:13.011442 13239 leader_election.cc:288] T 00000000000000000000000000000000 P 0de9b9e1e08d477391490d612b574a33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:12:13.011576 13245 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0de9b9e1e08d477391490d612b574a33 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:13.011848 13245 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0de9b9e1e08d477391490d612b574a33 [term 1 LEADER]: Becoming Leader. State: Replica: 0de9b9e1e08d477391490d612b574a33, State: Running, Role: LEADER
I20260812 03:12:13.012228 13245 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0de9b9e1e08d477391490d612b574a33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0de9b9e1e08d477391490d612b574a33" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 36407 } }
I20260812 03:12:13.012866 13239 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0de9b9e1e08d477391490d612b574a33 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:13.013619 13245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0de9b9e1e08d477391490d612b574a33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0de9b9e1e08d477391490d612b574a33. Latest consensus state: current_term: 1 leader_uuid: "0de9b9e1e08d477391490d612b574a33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0de9b9e1e08d477391490d612b574a33" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 36407 } } }
I20260812 03:12:13.013746 13245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0de9b9e1e08d477391490d612b574a33 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:13.014102 13247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0de9b9e1e08d477391490d612b574a33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0de9b9e1e08d477391490d612b574a33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0de9b9e1e08d477391490d612b574a33" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 36407 } } }
I20260812 03:12:13.014185 13256 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:12:13.014212 13247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0de9b9e1e08d477391490d612b574a33 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:13.014417 13243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:12.999053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53460 (local address 127.26.250.190:36407)
0812 03:12:12.999540 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:12.999556 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:12.999594 (+ 38us) server_negotiation.cc:407] Connection header received
0812 03:12:13.000329 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:13.000349 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:13.000747 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:13.001142 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:13.002031 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.003127 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.004038 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.004540 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.004728 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:13.007528 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:13.007534 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:13.007551 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:13.007583 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:13.009821 (+ 2238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:13.010497 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:13.010510 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:13.010516 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:13.010646 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:13.011054 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:13.011057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:13.011058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:13.011780 (+ 722us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:13.012752 (+ 972us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:13.012916 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 03:12:13.013250 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":158,"threads_started":1}
I20260812 03:12:13.017684 13256 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:12:13.027083 13256 catalog_manager.cc:1347] Generated new cluster ID: c3f0ee16e82c4dc79bfb1296c3dce008
I20260812 03:12:13.027192 13256 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:12:13.042132 13256 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:12:13.043211 13256 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:12:13.048928 13256 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0de9b9e1e08d477391490d612b574a33: Generated new TSK 0
I20260812 03:12:13.049674 13256 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.250.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
I20260812 03:12:13.085793 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:0
--local_ip_for_outbound_sockets=127.26.250.129
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:36407
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:46073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:13.216758 13269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:13.217047 13269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:13.217132 13269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:13.221345 13269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:13.221421 13269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:13.221527 13269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:12:13.227254 13269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:46073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:36407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:13.228493 13269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:13.229650 13269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:13.240336 13275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:13.240381 13277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:13.240447 13269 server_base.cc:1034] running on GCE node
W20260812 03:12:13.240357 13279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:13.241205 13269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:13.241977 13269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:13.243166 13269 hybrid_clock.cc:648] HybridClock initialized: now 1786504333243118 us; error 63 us; skew 500 ppm
Aug 12 03:12:13 dist-test-slave-zpfg krb5kdc[13153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504333, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:13.246064 13269 init.cc:377] Logged in from keytab as kudu/127.26.250.129@KRBTEST.COM (short username kudu)
I20260812 03:12:13.247033 13269 webserver.cc:460] Webserver started at http://127.26.250.129:40457/ using document root <none> and password file <none>
I20260812 03:12:13.247577 13269 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:13.247650 13269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:13.247821 13269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:13.249907 13269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-0/data/instance:
uuid: "ce14523288d74a32861b6af491a432a8"
format_stamp: "Formatted at 2026-08-12 03:12:13 on dist-test-slave-zpfg"
server_key: "8a2a5dbe14b1076d5355b62e42f758a5"
server_key_iv: "7c5f637ab898e5b691e3c35f30b8394e"
server_key_version: "encryptionkey@0"
I20260812 03:12:13.250536 13269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance:
uuid: "ce14523288d74a32861b6af491a432a8"
format_stamp: "Formatted at 2026-08-12 03:12:13 on dist-test-slave-zpfg"
server_key: "8a2a5dbe14b1076d5355b62e42f758a5"
server_key_iv: "7c5f637ab898e5b691e3c35f30b8394e"
server_key_version: "encryptionkey@0"
I20260812 03:12:13.254612 13269 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:12:13.257344 13287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:13.258740 13269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:12:13.258890 13269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "ce14523288d74a32861b6af491a432a8"
format_stamp: "Formatted at 2026-08-12 03:12:13 on dist-test-slave-zpfg"
server_key: "8a2a5dbe14b1076d5355b62e42f758a5"
server_key_iv: "7c5f637ab898e5b691e3c35f30b8394e"
server_key_version: "encryptionkey@0"
I20260812 03:12:13.259020 13269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:13.274626 13269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:13.278298 13269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:13.278559 13269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:13.279232 13269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:12:13.280238 13269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:12:13.280313 13269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:13.280376 13269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:12:13.280429 13269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:13.290311 13269 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:46597
I20260812 03:12:13.290486 13408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:46597 every 8 connection(s)
I20260812 03:12:13.291371 13269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:12:13.294305 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 13269
I20260812 03:12:13.294523 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance
I20260812 03:12:13.294878 27626 external_mini_cluster.cc:1428] Setting key a00077943e9b2d47797f9c0468dd728f
Aug 12 03:12:13 dist-test-slave-zpfg krb5kdc[13153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504333, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.250.130@KRBTEST.COM; defaulting to no policy
I20260812 03:12:13.308630 13243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:13.293415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:44129 (local address 127.26.250.190:36407)
0812 03:12:13.293568 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:13.293572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:13.294651 (+ 1079us) server_negotiation.cc:407] Connection header received
0812 03:12:13.296271 (+ 1620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:13.296277 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:13.296384 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:13.296488 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:13.298260 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.299067 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.300718 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.301147 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.301281 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:13.304210 (+ 2929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:13.304217 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:13.304222 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:13.304262 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:13.306076 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:13.306801 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:13.306804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:13.306806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:13.306892 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:13.307235 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:13.307237 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:13.307238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:13.307476 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:13.307633 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:13.308240 (+ 607us) server_negotiation.cc:299] Negotiation successful
0812 03:12:13.308404 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.130@KRBTEST.COM" created.
I20260812 03:12:13.309504 13411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:13.293886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:36407 (local address 127.26.250.129:44129)
0812 03:12:13.294461 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:13.294521 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:13.295987 (+ 1466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:13.296723 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:13.296733 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:13.297304 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:13.298072 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:13.298096 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.299374 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.299377 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:13.300564 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:13.300576 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.301382 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.301385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:13.301555 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:13.302331 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:13.302415 (+ 84us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:13.303996 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:13.306223 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:13.306229 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:13.306245 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:13.306684 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:13.306985 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:13.306987 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:13.306988 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:13.307136 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:13.307633 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:13.307642 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:13.307974 (+ 332us) client_negotiation.cc:769] Sending connection context
0812 03:12:13.308181 (+ 207us) client_negotiation.cc:240] Negotiation successful
0812 03:12:13.308567 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":172,"threads_started":1}
I20260812 03:12:13.311151 13409 heartbeater.cc:347] Connected to a master server at 127.26.250.190:36407
I20260812 03:12:13.311599 13409 heartbeater.cc:464] Registering TS with master...
I20260812 03:12:13.312403 13409 heartbeater.cc:511] Master 127.26.250.190:36407 requested a full tablet report, sending...
I20260812 03:12:13.314634 13200 ts_manager.cc:194] Registered new tserver with Master: ce14523288d74a32861b6af491a432a8 (127.26.250.129:46597)
I20260812 03:12:13.316454 13200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.129@KRBTEST.COM'} at 127.26.250.129:44129
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
I20260812 03:12:13.329401 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:0
--local_ip_for_outbound_sockets=127.26.250.130
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:36407
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:46073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:13.450817 13416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:13.451118 13416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:13.451172 13416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:13.455224 13416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:13.455289 13416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:13.455353 13416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:12:13.461004 13416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:46073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:36407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:13.462249 13416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:13.463515 13416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:13.474615 13421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:13.474766 13416 server_base.cc:1034] running on GCE node
W20260812 03:12:13.474615 13422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:13.474615 13424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:13.475419 13416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:13.476168 13416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:13.477452 13416 hybrid_clock.cc:648] HybridClock initialized: now 1786504333477431 us; error 31 us; skew 500 ppm
Aug 12 03:12:13 dist-test-slave-zpfg krb5kdc[13153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504333, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:13.481199 13416 init.cc:377] Logged in from keytab as kudu/127.26.250.130@KRBTEST.COM (short username kudu)
I20260812 03:12:13.482465 13416 webserver.cc:460] Webserver started at http://127.26.250.130:38969/ using document root <none> and password file <none>
I20260812 03:12:13.483204 13416 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:13.483284 13416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:13.483551 13416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:13.486037 13416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-1/data/instance:
uuid: "be37c2d9a6884d12b7ba10a50d6431e8"
format_stamp: "Formatted at 2026-08-12 03:12:13 on dist-test-slave-zpfg"
server_key: "face2b0aea33ec79ddb5ea9e01a5fb26"
server_key_iv: "1f1a6acea1b35b8878aef5d4f499b3f9"
server_key_version: "encryptionkey@0"
I20260812 03:12:13.486809 13416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance:
uuid: "be37c2d9a6884d12b7ba10a50d6431e8"
format_stamp: "Formatted at 2026-08-12 03:12:13 on dist-test-slave-zpfg"
server_key: "face2b0aea33ec79ddb5ea9e01a5fb26"
server_key_iv: "1f1a6acea1b35b8878aef5d4f499b3f9"
server_key_version: "encryptionkey@0"
I20260812 03:12:13.491688 13416 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:12:13.494957 13434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:13.496620 13416 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:12:13.496760 13416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "be37c2d9a6884d12b7ba10a50d6431e8"
format_stamp: "Formatted at 2026-08-12 03:12:13 on dist-test-slave-zpfg"
server_key: "face2b0aea33ec79ddb5ea9e01a5fb26"
server_key_iv: "1f1a6acea1b35b8878aef5d4f499b3f9"
server_key_version: "encryptionkey@0"
I20260812 03:12:13.496959 13416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:13.533389 13416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:13.537297 13416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:13.537567 13416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:13.538318 13416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:12:13.539402 13416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:12:13.539492 13416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:13.539572 13416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:12:13.539633 13416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:13.551000 13416 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:46317
I20260812 03:12:13.551041 13552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:46317 every 8 connection(s)
I20260812 03:12:13.552068 13416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:12:13.558508 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 13416
I20260812 03:12:13.558663 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance
I20260812 03:12:13.559048 27626 external_mini_cluster.cc:1428] Setting key d0e40120c019c653f79fc0b42b8fd10c
Aug 12 03:12:13 dist-test-slave-zpfg krb5kdc[13153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504333, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:12:13.568500 13243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:13.554307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:55277 (local address 127.26.250.190:36407)
0812 03:12:13.554516 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:13.554520 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:13.555642 (+ 1122us) server_negotiation.cc:407] Connection header received
0812 03:12:13.557140 (+ 1498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:13.557143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:13.557203 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:13.557303 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:13.558904 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.559558 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.561069 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.561322 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.561441 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:13.564589 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:13.564592 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:13.564595 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:13.564621 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:13.566153 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:13.566802 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:13.566805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:13.566806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:13.566888 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:13.567267 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:13.567271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:13.567272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:13.567478 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:13.567597 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:13.568172 (+ 575us) server_negotiation.cc:299] Negotiation successful
0812 03:12:13.568326 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260812 03:12:13.569514 13555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:13.554791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:36407 (local address 127.26.250.130:55277)
0812 03:12:13.555459 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:13.555510 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:13.556847 (+ 1337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:13.557504 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:13.557511 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:13.557963 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:13.558719 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:13.558746 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.559712 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.559715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:13.560936 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:13.560947 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.561493 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.561498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:13.561756 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:13.562551 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:13.562570 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:13.564386 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:13.566305 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:13.566311 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:13.566329 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:13.566658 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:13.566990 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:13.566993 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:13.566995 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:13.567158 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:13.567599 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:13.567609 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:13.567905 (+ 296us) client_negotiation.cc:769] Sending connection context
0812 03:12:13.568107 (+ 202us) client_negotiation.cc:240] Negotiation successful
0812 03:12:13.568505 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":164,"threads_started":1}
WARNING: no policy specified for kudu/127.26.250.131@KRBTEST.COM; defaulting to no policy
I20260812 03:12:13.571100 13553 heartbeater.cc:347] Connected to a master server at 127.26.250.190:36407
I20260812 03:12:13.571511 13553 heartbeater.cc:464] Registering TS with master...
I20260812 03:12:13.572134 13553 heartbeater.cc:511] Master 127.26.250.190:36407 requested a full tablet report, sending...
I20260812 03:12:13.573426 13200 ts_manager.cc:194] Registered new tserver with Master: be37c2d9a6884d12b7ba10a50d6431e8 (127.26.250.130:46317)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.131@KRBTEST.COM" created.
I20260812 03:12:13.574124 13200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.130@KRBTEST.COM'} at 127.26.250.130:55277
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
I20260812 03:12:13.594224 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:0
--local_ip_for_outbound_sockets=127.26.250.131
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:36407
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:46073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:13.717682 13558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:13.717976 13558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:13.718021 13558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:13.722290 13558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:13.722385 13558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:13.722448 13558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:12:13.728082 13558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:46073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:36407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:13.729239 13558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:13.730446 13558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:13.741039 13564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:13.741042 13567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:13.741117 13565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:13.741117 13558 server_base.cc:1034] running on GCE node
I20260812 03:12:13.741957 13558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:13.742638 13558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:13.743808 13558 hybrid_clock.cc:648] HybridClock initialized: now 1786504333743797 us; error 35 us; skew 500 ppm
Aug 12 03:12:13 dist-test-slave-zpfg krb5kdc[13153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504333, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:13.746649 13558 init.cc:377] Logged in from keytab as kudu/127.26.250.131@KRBTEST.COM (short username kudu)
I20260812 03:12:13.747550 13558 webserver.cc:460] Webserver started at http://127.26.250.131:41215/ using document root <none> and password file <none>
I20260812 03:12:13.748103 13558 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:13.748164 13558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:13.748339 13558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:13.750452 13558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-2/data/instance:
uuid: "630593549b88459aab32ab1313df8948"
format_stamp: "Formatted at 2026-08-12 03:12:13 on dist-test-slave-zpfg"
server_key: "7fecd56fa23df908b9aa235ae2d1c54c"
server_key_iv: "b2b63630298c251f656d1f9cb0f5d944"
server_key_version: "encryptionkey@0"
I20260812 03:12:13.750998 13558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance:
uuid: "630593549b88459aab32ab1313df8948"
format_stamp: "Formatted at 2026-08-12 03:12:13 on dist-test-slave-zpfg"
server_key: "7fecd56fa23df908b9aa235ae2d1c54c"
server_key_iv: "b2b63630298c251f656d1f9cb0f5d944"
server_key_version: "encryptionkey@0"
I20260812 03:12:13.755038 13558 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:12:13.757627 13574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:13.758996 13558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:12:13.759099 13558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "630593549b88459aab32ab1313df8948"
format_stamp: "Formatted at 2026-08-12 03:12:13 on dist-test-slave-zpfg"
server_key: "7fecd56fa23df908b9aa235ae2d1c54c"
server_key_iv: "b2b63630298c251f656d1f9cb0f5d944"
server_key_version: "encryptionkey@0"
I20260812 03:12:13.759207 13558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:13.769765 13558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:13.772948 13558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:13.773178 13558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:13.773808 13558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:12:13.774799 13558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:12:13.774856 13558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:13.774897 13558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:12:13.774914 13558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:13.784328 13558 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:34199
I20260812 03:12:13.784426 13689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:34199 every 8 connection(s)
I20260812 03:12:13.785302 13558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:12:13.792707 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 13558
I20260812 03:12:13.792860 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance
I20260812 03:12:13.793187 27626 external_mini_cluster.cc:1428] Setting key 55c6ff458817d32293800970c8fbef66
Aug 12 03:12:13 dist-test-slave-zpfg krb5kdc[13153](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504333, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:12:13.800112 13243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:13.787297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:53035 (local address 127.26.250.190:36407)
0812 03:12:13.787516 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:13.787522 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:13.788455 (+ 933us) server_negotiation.cc:407] Connection header received
0812 03:12:13.789825 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:13.789828 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:13.789878 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:13.789958 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:13.791251 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.791823 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.793158 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.793446 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.793568 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:13.796732 (+ 3164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:13.796735 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:13.796737 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:13.796763 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:13.798083 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:13.798661 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:13.798664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:13.798665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:13.798744 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:13.799062 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:13.799065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:13.799066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:13.799254 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:13.799327 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:13.799872 (+ 545us) server_negotiation.cc:299] Negotiation successful
0812 03:12:13.799982 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260812 03:12:13.801100 13692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:13.787778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:36407 (local address 127.26.250.131:53035)
0812 03:12:13.788280 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:13.788335 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:13.789615 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:13.790137 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:13.790144 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:13.790546 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:13.791098 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:13.791112 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.791961 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.791964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:13.793026 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:13.793037 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.793617 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.793621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:13.793840 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:13.794690 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:13.794732 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:13.796569 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:13.798192 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:13.798197 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:13.798211 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:13.798571 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:13.798828 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:13.798830 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:13.798832 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:13.798974 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:13.799376 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:13.799381 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:13.799641 (+ 260us) client_negotiation.cc:769] Sending connection context
0812 03:12:13.799797 (+ 156us) client_negotiation.cc:240] Negotiation successful
0812 03:12:13.800076 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":124,"threads_started":1}
I20260812 03:12:13.802501 13690 heartbeater.cc:347] Connected to a master server at 127.26.250.190:36407
I20260812 03:12:13.802776 13690 heartbeater.cc:464] Registering TS with master...
I20260812 03:12:13.803319 13690 heartbeater.cc:511] Master 127.26.250.190:36407 requested a full tablet report, sending...
I20260812 03:12:13.804422 13200 ts_manager.cc:194] Registered new tserver with Master: 630593549b88459aab32ab1313df8948 (127.26.250.131:34199)
I20260812 03:12:13.805011 13200 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.131@KRBTEST.COM'} at 127.26.250.131:53035
I20260812 03:12:13.808033 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:12:13.818688 13243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:13.810324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53470 (local address 127.26.250.190:36407)
0812 03:12:13.810527 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:13.810532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:13.810714 (+ 182us) server_negotiation.cc:407] Connection header received
0812 03:12:13.810914 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:13.810917 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:13.810969 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:13.811069 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:13.811864 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.812400 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.813368 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.813599 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.813709 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:13.815480 (+ 1771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:13.815483 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:13.815485 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:13.815510 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:13.816847 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:13.817301 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:13.817306 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:13.817312 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:13.817388 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:13.817669 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:13.817673 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:13.817675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:13.817948 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:13.818093 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:13.818382 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 03:12:13.818510 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260812 03:12:13.822096 13200 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53470:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:12:13.825397 13200 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:12:13.840643 13705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:13.834631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:34199 (local address 127.0.0.1:60956)
0812 03:12:13.835507 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:13.835522 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:13.835612 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:13.836472 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:13.836475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:13.836495 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:13.836600 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:13.836605 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.837920 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.837923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:13.838963 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:13.838974 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.840009 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.840012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:13.840123 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:13.840161 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 03:12:13.840309 (+ 148us) client_negotiation.cc:240] Negotiation successful
0812 03:12:13.840449 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"mutex_wait_us":58,"thread_start_us":130,"threads_started":1}
I20260812 03:12:13.840621 13707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:13.835263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60956 (local address 127.26.250.131:34199)
0812 03:12:13.835771 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:13.835781 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:13.835808 (+ 27us) server_negotiation.cc:407] Connection header received
0812 03:12:13.835890 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:13.835896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:13.836119 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:13.836294 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:13.836723 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.837750 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.839110 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.839867 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.840012 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:13.840166 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:13.840349 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 03:12:13.840456 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":190,"threads_started":1}
I20260812 03:12:13.840968 13706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:13.835197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:46597 (local address 127.0.0.1:56722)
0812 03:12:13.835781 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:13.835794 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:13.835916 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:13.836878 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:13.836881 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:13.836898 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:13.836977 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:13.836982 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.838086 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.838090 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:13.839285 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:13.839295 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.840577 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.840580 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:13.840663 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:13.840684 (+ 21us) client_negotiation.cc:769] Sending connection context
0812 03:12:13.840756 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 03:12:13.840855 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"mutex_wait_us":28,"thread_start_us":80,"threads_started":1}
I20260812 03:12:13.841149 13703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:13.834377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:46317 (local address 127.0.0.1:38474)
0812 03:12:13.834760 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:13.834831 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:13.834986 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:13.835816 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:13.835822 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:13.835847 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:13.836013 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:13.836021 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.837449 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.837453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:13.838884 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:13.838895 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.840236 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.840240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:13.840352 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:13.840374 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 03:12:13.840441 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 03:12:13.840516 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":87,"threads_started":1}
I20260812 03:12:13.841459 13708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:13.835395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56722 (local address 127.26.250.129:46597)
0812 03:12:13.836249 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:13.836255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:13.836276 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:12:13.836398 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:13.836406 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:13.836593 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:13.836730 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:13.837098 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.837966 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.839426 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.840268 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.840384 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:13.840515 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:13.841163 (+ 648us) server_negotiation.cc:299] Negotiation successful
0812 03:12:13.841290 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":744,"thread_start_us":97,"threads_started":1}
I20260812 03:12:13.841465 13704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:13.834533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38474 (local address 127.26.250.130:46317)
0812 03:12:13.835075 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:13.835088 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:13.835114 (+ 26us) server_negotiation.cc:407] Connection header received
0812 03:12:13.835198 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:13.835203 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:13.835433 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:13.835605 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:13.836179 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.837292 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.839032 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:13.839915 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:13.840041 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:13.840165 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:13.841216 (+ 1051us) server_negotiation.cc:299] Negotiation successful
0812 03:12:13.841325 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":180,"threads_started":1}
I20260812 03:12:13.844038 13339 tablet_service.cc:1467] Processing CreateTablet for tablet f938b1993e3945dda3db69444f921f57 (DEFAULT_TABLE table=test-table [id=45ccc913ed514882a512e4405802c7f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:12:13.844069 13624 tablet_service.cc:1467] Processing CreateTablet for tablet f938b1993e3945dda3db69444f921f57 (DEFAULT_TABLE table=test-table [id=45ccc913ed514882a512e4405802c7f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:12:13.844342 13485 tablet_service.cc:1467] Processing CreateTablet for tablet f938b1993e3945dda3db69444f921f57 (DEFAULT_TABLE table=test-table [id=45ccc913ed514882a512e4405802c7f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:12:13.845146 13339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f938b1993e3945dda3db69444f921f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:13.845211 13624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f938b1993e3945dda3db69444f921f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:13.845222 13485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f938b1993e3945dda3db69444f921f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:13.853206 13711 tablet_bootstrap.cc:492] T f938b1993e3945dda3db69444f921f57 P be37c2d9a6884d12b7ba10a50d6431e8: Bootstrap starting.
I20260812 03:12:13.853206 13712 tablet_bootstrap.cc:492] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948: Bootstrap starting.
I20260812 03:12:13.855824 13711 tablet_bootstrap.cc:654] T f938b1993e3945dda3db69444f921f57 P be37c2d9a6884d12b7ba10a50d6431e8: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:13.855823 13712 tablet_bootstrap.cc:654] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:13.856382 13709 tablet_bootstrap.cc:492] T f938b1993e3945dda3db69444f921f57 P ce14523288d74a32861b6af491a432a8: Bootstrap starting.
I20260812 03:12:13.857319 13712 log.cc:824] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:13.857311 13711 log.cc:824] T f938b1993e3945dda3db69444f921f57 P be37c2d9a6884d12b7ba10a50d6431e8: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:13.858815 13709 tablet_bootstrap.cc:654] T f938b1993e3945dda3db69444f921f57 P ce14523288d74a32861b6af491a432a8: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:13.859522 13712 tablet_bootstrap.cc:492] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948: No bootstrap required, opened a new log
I20260812 03:12:13.859587 13711 tablet_bootstrap.cc:492] T f938b1993e3945dda3db69444f921f57 P be37c2d9a6884d12b7ba10a50d6431e8: No bootstrap required, opened a new log
I20260812 03:12:13.859715 13709 log.cc:824] T f938b1993e3945dda3db69444f921f57 P ce14523288d74a32861b6af491a432a8: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:13.859750 13712 ts_tablet_manager.cc:1397] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 03:12:13.859783 13711 ts_tablet_manager.cc:1397] T f938b1993e3945dda3db69444f921f57 P be37c2d9a6884d12b7ba10a50d6431e8: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 03:12:13.861930 13709 tablet_bootstrap.cc:492] T f938b1993e3945dda3db69444f921f57 P ce14523288d74a32861b6af491a432a8: No bootstrap required, opened a new log
I20260812 03:12:13.862135 13709 ts_tablet_manager.cc:1397] T f938b1993e3945dda3db69444f921f57 P ce14523288d74a32861b6af491a432a8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:12:13.863428 13712 raft_consensus.cc:348] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be37c2d9a6884d12b7ba10a50d6431e8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46317 } } peers { permanent_uuid: "ce14523288d74a32861b6af491a432a8" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46597 } } peers { permanent_uuid: "630593549b88459aab32ab1313df8948" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 34199 } }
I20260812 03:12:13.863709 13712 raft_consensus.cc:374] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:13.863771 13712 raft_consensus.cc:729] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 630593549b88459aab32ab1313df8948, State: Initialized, Role: FOLLOWER
I20260812 03:12:13.864373 13712 consensus_queue.cc:260] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be37c2d9a6884d12b7ba10a50d6431e8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46317 } } peers { permanent_uuid: "ce14523288d74a32861b6af491a432a8" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46597 } } peers { permanent_uuid: "630593549b88459aab32ab1313df8948" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 34199 } }
I20260812 03:12:13.864697 13711 raft_consensus.cc:348] T f938b1993e3945dda3db69444f921f57 P be37c2d9a6884d12b7ba10a50d6431e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be37c2d9a6884d12b7ba10a50d6431e8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46317 } } peers { permanent_uuid: "ce14523288d74a32861b6af491a432a8" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46597 } } peers { permanent_uuid: "630593549b88459aab32ab1313df8948" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 34199 } }
I20260812 03:12:13.864984 13711 raft_consensus.cc:374] T f938b1993e3945dda3db69444f921f57 P be37c2d9a6884d12b7ba10a50d6431e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:13.865034 13711 raft_consensus.cc:729] T f938b1993e3945dda3db69444f921f57 P be37c2d9a6884d12b7ba10a50d6431e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be37c2d9a6884d12b7ba10a50d6431e8, State: Initialized, Role: FOLLOWER
I20260812 03:12:13.865502 13712 ts_tablet_manager.cc:1428] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260812 03:12:13.865480 13711 consensus_queue.cc:260] T f938b1993e3945dda3db69444f921f57 P be37c2d9a6884d12b7ba10a50d6431e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be37c2d9a6884d12b7ba10a50d6431e8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46317 } } peers { permanent_uuid: "ce14523288d74a32861b6af491a432a8" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46597 } } peers { permanent_uuid: "630593549b88459aab32ab1313df8948" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 34199 } }
I20260812 03:12:13.865914 13690 heartbeater.cc:503] Master 127.26.250.190:36407 was elected leader, sending a full tablet report...
I20260812 03:12:13.866407 13553 heartbeater.cc:503] Master 127.26.250.190:36407 was elected leader, sending a full tablet report...
I20260812 03:12:13.866523 13711 ts_tablet_manager.cc:1428] T f938b1993e3945dda3db69444f921f57 P be37c2d9a6884d12b7ba10a50d6431e8: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 03:12:13.866798 13709 raft_consensus.cc:348] T f938b1993e3945dda3db69444f921f57 P ce14523288d74a32861b6af491a432a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be37c2d9a6884d12b7ba10a50d6431e8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46317 } } peers { permanent_uuid: "ce14523288d74a32861b6af491a432a8" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46597 } } peers { permanent_uuid: "630593549b88459aab32ab1313df8948" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 34199 } }
I20260812 03:12:13.866999 13709 raft_consensus.cc:374] T f938b1993e3945dda3db69444f921f57 P ce14523288d74a32861b6af491a432a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:13.867076 13709 raft_consensus.cc:729] T f938b1993e3945dda3db69444f921f57 P ce14523288d74a32861b6af491a432a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce14523288d74a32861b6af491a432a8, State: Initialized, Role: FOLLOWER
I20260812 03:12:13.867527 13709 consensus_queue.cc:260] T f938b1993e3945dda3db69444f921f57 P ce14523288d74a32861b6af491a432a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be37c2d9a6884d12b7ba10a50d6431e8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46317 } } peers { permanent_uuid: "ce14523288d74a32861b6af491a432a8" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46597 } } peers { permanent_uuid: "630593549b88459aab32ab1313df8948" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 34199 } }
I20260812 03:12:13.868184 13409 heartbeater.cc:503] Master 127.26.250.190:36407 was elected leader, sending a full tablet report...
I20260812 03:12:13.868539 13709 ts_tablet_manager.cc:1428] T f938b1993e3945dda3db69444f921f57 P ce14523288d74a32861b6af491a432a8: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
W20260812 03:12:14.036414 13691 tablet.cc:2369] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:12:14.042984 13410 tablet.cc:2369] T f938b1993e3945dda3db69444f921f57 P ce14523288d74a32861b6af491a432a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:12:14.053495 13554 tablet.cc:2369] T f938b1993e3945dda3db69444f921f57 P be37c2d9a6884d12b7ba10a50d6431e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:12:14.121498 13716 raft_consensus.cc:482] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:12:14.121769 13716 raft_consensus.cc:504] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be37c2d9a6884d12b7ba10a50d6431e8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46317 } } peers { permanent_uuid: "ce14523288d74a32861b6af491a432a8" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46597 } } peers { permanent_uuid: "630593549b88459aab32ab1313df8948" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 34199 } }
I20260812 03:12:14.123179 13716 leader_election.cc:288] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be37c2d9a6884d12b7ba10a50d6431e8 (127.26.250.130:46317), ce14523288d74a32861b6af491a432a8 (127.26.250.129:46597)
I20260812 03:12:14.126613 13717 raft_consensus.cc:482] T f938b1993e3945dda3db69444f921f57 P be37c2d9a6884d12b7ba10a50d6431e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:12:14.126960 13717 raft_consensus.cc:504] T f938b1993e3945dda3db69444f921f57 P be37c2d9a6884d12b7ba10a50d6431e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be37c2d9a6884d12b7ba10a50d6431e8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46317 } } peers { permanent_uuid: "ce14523288d74a32861b6af491a432a8" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46597 } } peers { permanent_uuid: "630593549b88459aab32ab1313df8948" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 34199 } }
I20260812 03:12:14.127314 13692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:14.123433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:46317 (local address 127.26.250.131:41943)
0812 03:12:14.123613 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:14.123634 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:14.123770 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:14.124088 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:14.124091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:14.124108 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:14.124243 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:14.124281 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.125126 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.125133 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:14.125955 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:14.125964 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.126822 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.126826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:14.126969 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:14.127000 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 03:12:14.127088 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 03:12:14.127169 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 03:12:14.127365 13704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:14.123604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:41943 (local address 127.26.250.130:46317)
0812 03:12:14.123784 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:14.123790 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:14.123811 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:12:14.123901 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:14.123907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:14.123970 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:14.124056 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:14.124438 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.124998 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.126102 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.126662 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.126822 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:14.126877 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:14.127143 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 03:12:14.127231 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 03:12:14.128026 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f938b1993e3945dda3db69444f921f57" candidate_uuid: "630593549b88459aab32ab1313df8948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be37c2d9a6884d12b7ba10a50d6431e8" is_pre_election: true
I20260812 03:12:14.128291 13505 raft_consensus.cc:2454] T f938b1993e3945dda3db69444f921f57 P be37c2d9a6884d12b7ba10a50d6431e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 630593549b88459aab32ab1313df8948 in term 0.
I20260812 03:12:14.128391 13708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:14.123976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:52561 (local address 127.26.250.129:46597)
0812 03:12:14.124117 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:14.124125 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:14.124235 (+ 110us) server_negotiation.cc:407] Connection header received
0812 03:12:14.124457 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:14.124460 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:14.124511 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:14.124608 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:14.124922 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.125523 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.127011 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.127647 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.127764 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:14.127824 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:14.128150 (+ 326us) server_negotiation.cc:299] Negotiation successful
0812 03:12:14.128245 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 03:12:14.128777 13719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:14.123733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:46597 (local address 127.26.250.131:52561)
0812 03:12:14.124113 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:14.124129 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:14.124252 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:14.124622 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:14.124625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:14.124642 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:14.124798 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:14.124804 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.125640 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.125647 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:14.126811 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:14.126823 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.127889 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.127893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:14.127991 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:14.128017 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 03:12:14.128538 (+ 521us) client_negotiation.cc:240] Negotiation successful
0812 03:12:14.128638 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":177,"threads_started":1}
I20260812 03:12:14.128840 13575 leader_election.cc:302] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 630593549b88459aab32ab1313df8948, be37c2d9a6884d12b7ba10a50d6431e8; no voters:
I20260812 03:12:14.129307 13716 raft_consensus.cc:2781] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:12:14.129419 13716 raft_consensus.cc:482] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:12:14.129487 13716 raft_consensus.cc:3037] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:14.129676 13360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f938b1993e3945dda3db69444f921f57" candidate_uuid: "630593549b88459aab32ab1313df8948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce14523288d74a32861b6af491a432a8" is_pre_election: true
I20260812 03:12:14.130007 13717 leader_election.cc:288] T f938b1993e3945dda3db69444f921f57 P be37c2d9a6884d12b7ba10a50d6431e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce14523288d74a32861b6af491a432a8 (127.26.250.129:46597), 630593549b88459aab32ab1313df8948 (127.26.250.131:34199)
I20260812 03:12:14.130031 13360 raft_consensus.cc:2454] T f938b1993e3945dda3db69444f921f57 P ce14523288d74a32861b6af491a432a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 630593549b88459aab32ab1313df8948 in term 0.
I20260812 03:12:14.131852 13716 raft_consensus.cc:504] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be37c2d9a6884d12b7ba10a50d6431e8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46317 } } peers { permanent_uuid: "ce14523288d74a32861b6af491a432a8" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46597 } } peers { permanent_uuid: "630593549b88459aab32ab1313df8948" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 34199 } }
I20260812 03:12:14.132225 13716 leader_election.cc:288] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948 [CANDIDATE]: Term 1 election: Requested vote from peers be37c2d9a6884d12b7ba10a50d6431e8 (127.26.250.130:46317), ce14523288d74a32861b6af491a432a8 (127.26.250.129:46597)
I20260812 03:12:14.132606 13505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f938b1993e3945dda3db69444f921f57" candidate_uuid: "630593549b88459aab32ab1313df8948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be37c2d9a6884d12b7ba10a50d6431e8"
I20260812 03:12:14.132686 13360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f938b1993e3945dda3db69444f921f57" candidate_uuid: "630593549b88459aab32ab1313df8948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce14523288d74a32861b6af491a432a8"
I20260812 03:12:14.132735 13505 raft_consensus.cc:3037] T f938b1993e3945dda3db69444f921f57 P be37c2d9a6884d12b7ba10a50d6431e8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:14.132825 13360 raft_consensus.cc:3037] T f938b1993e3945dda3db69444f921f57 P ce14523288d74a32861b6af491a432a8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:14.134089 13708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:14.129153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:51319 (local address 127.26.250.129:46597)
0812 03:12:14.129292 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:14.129296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:14.129615 (+ 319us) server_negotiation.cc:407] Connection header received
0812 03:12:14.129846 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:14.129851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:14.129902 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:14.129976 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:14.130414 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.131069 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.132644 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.133292 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.133428 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:14.133479 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:14.133867 (+ 388us) server_negotiation.cc:299] Negotiation successful
0812 03:12:14.133965 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 03:12:14.134313 13505 raft_consensus.cc:2454] T f938b1993e3945dda3db69444f921f57 P be37c2d9a6884d12b7ba10a50d6431e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 630593549b88459aab32ab1313df8948 in term 1.
I20260812 03:12:14.134409 13360 raft_consensus.cc:2454] T f938b1993e3945dda3db69444f921f57 P ce14523288d74a32861b6af491a432a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 630593549b88459aab32ab1313df8948 in term 1.
I20260812 03:12:14.134630 13720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:14.128810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:46597 (local address 127.26.250.130:51319)
0812 03:12:14.129373 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:14.129392 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:14.129700 (+ 308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:14.130081 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:14.130087 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:14.130110 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:14.130231 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:14.130238 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.131222 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.131225 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:14.132499 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:14.132509 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.133603 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.133607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:14.133702 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:14.133736 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 03:12:14.134294 (+ 558us) client_negotiation.cc:240] Negotiation successful
0812 03:12:14.134450 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":1,"thread_start_us":192,"threads_started":1}
I20260812 03:12:14.134732 13575 leader_election.cc:302] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 630593549b88459aab32ab1313df8948, be37c2d9a6884d12b7ba10a50d6431e8; no voters:
I20260812 03:12:14.135033 13716 raft_consensus.cc:2781] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:14.135190 13360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f938b1993e3945dda3db69444f921f57" candidate_uuid: "be37c2d9a6884d12b7ba10a50d6431e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce14523288d74a32861b6af491a432a8" is_pre_election: true
I20260812 03:12:14.135295 13707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:14.130390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:45649 (local address 127.26.250.131:34199)
0812 03:12:14.130547 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:14.130552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:14.130701 (+ 149us) server_negotiation.cc:407] Connection header received
0812 03:12:14.130850 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:14.130857 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:14.130907 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:14.131013 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:14.131300 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.131846 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.133449 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.134150 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.134286 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:14.134386 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:14.135060 (+ 674us) server_negotiation.cc:299] Negotiation successful
0812 03:12:14.135160 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 03:12:14.135354 13716 raft_consensus.cc:686] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948 [term 1 LEADER]: Becoming Leader. State: Replica: 630593549b88459aab32ab1313df8948, State: Running, Role: LEADER
I20260812 03:12:14.135419 13360 raft_consensus.cc:2379] T f938b1993e3945dda3db69444f921f57 P ce14523288d74a32861b6af491a432a8 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate be37c2d9a6884d12b7ba10a50d6431e8 in current term 1: Already voted for candidate 630593549b88459aab32ab1313df8948 in this term.
I20260812 03:12:14.135737 13721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:14.130210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:34199 (local address 127.26.250.130:45649)
0812 03:12:14.130586 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:14.130600 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:14.130731 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:14.131026 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:14.131029 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:14.131047 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:14.131180 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:14.131185 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.131971 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.131977 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:14.133290 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:14.133302 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.134506 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.134510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:14.134610 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:14.134643 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 03:12:14.135469 (+ 826us) client_negotiation.cc:240] Negotiation successful
0812 03:12:14.135558 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":134,"threads_started":1}
I20260812 03:12:14.135769 13716 consensus_queue.cc:237] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be37c2d9a6884d12b7ba10a50d6431e8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46317 } } peers { permanent_uuid: "ce14523288d74a32861b6af491a432a8" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46597 } } peers { permanent_uuid: "630593549b88459aab32ab1313df8948" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 34199 } }
I20260812 03:12:14.136372 13644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f938b1993e3945dda3db69444f921f57" candidate_uuid: "be37c2d9a6884d12b7ba10a50d6431e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "630593549b88459aab32ab1313df8948" is_pre_election: true
I20260812 03:12:14.137027 13436 leader_election.cc:302] T f938b1993e3945dda3db69444f921f57 P be37c2d9a6884d12b7ba10a50d6431e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: be37c2d9a6884d12b7ba10a50d6431e8; no voters: 630593549b88459aab32ab1313df8948, ce14523288d74a32861b6af491a432a8
I20260812 03:12:14.137416 13717 raft_consensus.cc:2726] T f938b1993e3945dda3db69444f921f57 P be37c2d9a6884d12b7ba10a50d6431e8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260812 03:12:14.140101 13200 catalog_manager.cc:5495] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948 reported cstate change: term changed from 0 to 1, leader changed from <none> to 630593549b88459aab32ab1313df8948 (127.26.250.131). New cstate: current_term: 1 leader_uuid: "630593549b88459aab32ab1313df8948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be37c2d9a6884d12b7ba10a50d6431e8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce14523288d74a32861b6af491a432a8" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "630593549b88459aab32ab1313df8948" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 34199 } health_report { overall_health: HEALTHY } } }
I20260812 03:12:14.209525 13707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:14.205380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60960 (local address 127.26.250.131:34199)
0812 03:12:14.205559 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:14.205563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:14.205578 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:12:14.205660 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:14.205662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:14.205716 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:14.205824 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:12:14.206141 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.206868 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.207777 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.208019 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.208111 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:14.208495 (+ 384us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:12:14.209075 (+ 580us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:12:14.209182 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:14.209258 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 03:12:14.209317 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260812 03:12:14.216756 13360 raft_consensus.cc:1261] T f938b1993e3945dda3db69444f921f57 P ce14523288d74a32861b6af491a432a8 [term 1 FOLLOWER]: Refusing update from remote peer 630593549b88459aab32ab1313df8948: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:12:14.216794 13505 raft_consensus.cc:1261] T f938b1993e3945dda3db69444f921f57 P be37c2d9a6884d12b7ba10a50d6431e8 [term 1 FOLLOWER]: Refusing update from remote peer 630593549b88459aab32ab1313df8948: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:12:14.217586 13722 consensus_queue.cc:1035] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be37c2d9a6884d12b7ba10a50d6431e8" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 46317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:12:14.217813 13716 consensus_queue.cc:1035] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce14523288d74a32861b6af491a432a8" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 46597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:12:14.225432 13726 mvcc.cc:205] Tried to move back new op lower bound from 7317521752943054848 to 7317521752626728960. Current Snapshot: MvccSnapshot[applied={T|T < 7317521752943054848}]
I20260812 03:12:14.235638 13200 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53470:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:12:14.235869 13200 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53470:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:12:14.238749 13200 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0de9b9e1e08d477391490d612b574a33: Sending DeleteTablet for 3 replicas of tablet f938b1993e3945dda3db69444f921f57
I20260812 03:12:14.239686 13624 tablet_service.cc:1514] Processing DeleteTablet for tablet f938b1993e3945dda3db69444f921f57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:12:14 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:60956
I20260812 03:12:14.239810 13485 tablet_service.cc:1514] Processing DeleteTablet for tablet f938b1993e3945dda3db69444f921f57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:12:14 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:38474
I20260812 03:12:14.239852 13339 tablet_service.cc:1514] Processing DeleteTablet for tablet f938b1993e3945dda3db69444f921f57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:12:14 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:56722
I20260812 03:12:14.240474 13738 tablet_replica.cc:330] T f938b1993e3945dda3db69444f921f57 P ce14523288d74a32861b6af491a432a8: stopping tablet replica
I20260812 03:12:14.240756 13736 tablet_replica.cc:330] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948: stopping tablet replica
I20260812 03:12:14.240882 13738 raft_consensus.cc:2229] T f938b1993e3945dda3db69444f921f57 P ce14523288d74a32861b6af491a432a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:12:14.241072 13735 tablet_replica.cc:330] T f938b1993e3945dda3db69444f921f57 P be37c2d9a6884d12b7ba10a50d6431e8: stopping tablet replica
I20260812 03:12:14.241258 13738 raft_consensus.cc:2258] T f938b1993e3945dda3db69444f921f57 P ce14523288d74a32861b6af491a432a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:12:14.241289 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 13269
I20260812 03:12:14.242889 13735 raft_consensus.cc:2229] T f938b1993e3945dda3db69444f921f57 P be37c2d9a6884d12b7ba10a50d6431e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:12:14.243212 13736 raft_consensus.cc:2229] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:12:14.243785 13736 raft_consensus.cc:2258] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:12:14.244228 13735 raft_consensus.cc:2258] T f938b1993e3945dda3db69444f921f57 P be37c2d9a6884d12b7ba10a50d6431e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:12:14.246234 13736 ts_tablet_manager.cc:1905] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:12:14.249336 13736 ts_tablet_manager.cc:1918] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:12:14.249459 13736 log.cc:1196] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/f938b1993e3945dda3db69444f921f57
I20260812 03:12:14.249825 13736 ts_tablet_manager.cc:1939] T f938b1993e3945dda3db69444f921f57 P 630593549b88459aab32ab1313df8948: Deleting consensus metadata
I20260812 03:12:14.251369 13735 ts_tablet_manager.cc:1905] T f938b1993e3945dda3db69444f921f57 P be37c2d9a6884d12b7ba10a50d6431e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:12:14.251439 13184 catalog_manager.cc:4841] TS 630593549b88459aab32ab1313df8948 (127.26.250.131:34199): tablet f938b1993e3945dda3db69444f921f57 (table test-table [id=45ccc913ed514882a512e4405802c7f4]) successfully deleted
W20260812 03:12:14.251916 13183 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.250.129:46597 (error 108)
I20260812 03:12:14.252216 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 13416
I20260812 03:12:14.255618 13703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:14.255256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:46597 (local address 127.0.0.1:56738)
0812 03:12:14.255424 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:14.255516 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.129:46597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260812 03:12:14.255919 13183 catalog_manager.cc:4584] TS ce14523288d74a32861b6af491a432a8 (127.26.250.129:46597): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f938b1993e3945dda3db69444f921f57: Network error: Client connection negotiation failed: client connection to 127.26.250.129:46597: connect: Connection refused (error 111)
I20260812 03:12:14.260568 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 13558
I20260812 03:12:14.262773 13703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:14.260784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:46317 (local address 127.0.0.1:38486)
0812 03:12:14.262573 (+ 1789us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:14.262664 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.130:46317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1670}
I20260812 03:12:14.269829 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 13167
2026-08-12T03:12:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2760 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T03:12:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:12:14Z Disabled control of system clock
I20260812 03:12:14.306308 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:39617
--webserver_interface=127.26.250.190
--webserver_port=0
--builtin_ntp_servers=127.26.250.148:33271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:39617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:12:14.429016 13743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:14.429327 13743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:14.429368 13743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:14.433614 13743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:14.433687 13743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:14.433704 13743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:14.433722 13743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:14.433737 13743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:14.438585 13743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:33271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:39617
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:39617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:14.439702 13743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:14.441109 13743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:14.450284 13749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:14.450299 13748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:14.450538 13743 server_base.cc:1034] running on GCE node
W20260812 03:12:14.450289 13751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:14.451277 13743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:14.452423 13743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:14.453599 13743 hybrid_clock.cc:648] HybridClock initialized: now 1786504334453580 us; error 32 us; skew 500 ppm
I20260812 03:12:14.455559 13743 webserver.cc:460] Webserver started at http://127.26.250.190:35815/ using document root <none> and password file <none>
I20260812 03:12:14.456139 13743 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:14.456205 13743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:14.456384 13743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:14.458631 13743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "1ee185397f66454292fd20c3c490d25c"
format_stamp: "Formatted at 2026-08-12 03:12:14 on dist-test-slave-zpfg"
server_key: "0ae52b3a3c26ae65324ee82cdeeed4eb"
server_key_iv: "cda1f54dd7ccd2b620954f31d01ab83c"
server_key_version: "encryptionkey@0"
I20260812 03:12:14.459183 13743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "1ee185397f66454292fd20c3c490d25c"
format_stamp: "Formatted at 2026-08-12 03:12:14 on dist-test-slave-zpfg"
server_key: "0ae52b3a3c26ae65324ee82cdeeed4eb"
server_key_iv: "cda1f54dd7ccd2b620954f31d01ab83c"
server_key_version: "encryptionkey@0"
I20260812 03:12:14.463091 13743 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:12:14.465698 13757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:14.467115 13743 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:12:14.467264 13743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "1ee185397f66454292fd20c3c490d25c"
format_stamp: "Formatted at 2026-08-12 03:12:14 on dist-test-slave-zpfg"
server_key: "0ae52b3a3c26ae65324ee82cdeeed4eb"
server_key_iv: "cda1f54dd7ccd2b620954f31d01ab83c"
server_key_version: "encryptionkey@0"
I20260812 03:12:14.467389 13743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:14.490526 13743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:14.491397 13743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:14.491626 13743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:14.499691 13743 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:39617
I20260812 03:12:14.499711 13810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:39617 every 8 connection(s)
I20260812 03:12:14.500739 13743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:12:14.503710 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 13743
I20260812 03:12:14.503821 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:12:14.504050 13812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:14.504179 27626 external_mini_cluster.cc:1428] Setting key 20cf0110160c844f1864c206f4c4fec1
I20260812 03:12:14.511757 13812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ee185397f66454292fd20c3c490d25c: Bootstrap starting.
I20260812 03:12:14.514765 13812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ee185397f66454292fd20c3c490d25c: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:14.515635 13815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:14.505804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38820 (local address 127.26.250.190:39617)
0812 03:12:14.506580 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:14.506599 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:14.506648 (+ 49us) server_negotiation.cc:407] Connection header received
0812 03:12:14.507698 (+ 1050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:14.507723 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:14.508288 (+ 565us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:14.508691 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:14.509156 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.510746 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.511671 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.512364 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.512527 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:14.513442 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:14.513449 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:14.513466 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:14.513960 (+ 494us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:14.514035 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:14.514063 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:14.514176 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:14.514291 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 03:12:14.514612 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":122,"threads_started":1}
I20260812 03:12:14.515700 13812 log.cc:824] T 00000000000000000000000000000000 P 1ee185397f66454292fd20c3c490d25c: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:14.517750 13812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ee185397f66454292fd20c3c490d25c: No bootstrap required, opened a new log
I20260812 03:12:14.521466 13812 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ee185397f66454292fd20c3c490d25c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee185397f66454292fd20c3c490d25c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39617 } }
I20260812 03:12:14.521754 13812 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ee185397f66454292fd20c3c490d25c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:14.521855 13812 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ee185397f66454292fd20c3c490d25c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ee185397f66454292fd20c3c490d25c, State: Initialized, Role: FOLLOWER
I20260812 03:12:14.522324 13812 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ee185397f66454292fd20c3c490d25c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee185397f66454292fd20c3c490d25c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39617 } }
I20260812 03:12:14.522521 13812 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ee185397f66454292fd20c3c490d25c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:12:14.522573 13812 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ee185397f66454292fd20c3c490d25c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:12:14.522663 13812 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ee185397f66454292fd20c3c490d25c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:14.523629 13812 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ee185397f66454292fd20c3c490d25c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee185397f66454292fd20c3c490d25c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39617 } }
I20260812 03:12:14.523952 13812 leader_election.cc:302] T 00000000000000000000000000000000 P 1ee185397f66454292fd20c3c490d25c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ee185397f66454292fd20c3c490d25c; no voters:
I20260812 03:12:14.524242 13812 leader_election.cc:288] T 00000000000000000000000000000000 P 1ee185397f66454292fd20c3c490d25c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:12:14.524480 13817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ee185397f66454292fd20c3c490d25c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:14.524716 13817 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ee185397f66454292fd20c3c490d25c [term 1 LEADER]: Becoming Leader. State: Replica: 1ee185397f66454292fd20c3c490d25c, State: Running, Role: LEADER
I20260812 03:12:14.525044 13817 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ee185397f66454292fd20c3c490d25c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee185397f66454292fd20c3c490d25c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39617 } }
I20260812 03:12:14.525708 13812 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ee185397f66454292fd20c3c490d25c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:14.526930 13818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ee185397f66454292fd20c3c490d25c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ee185397f66454292fd20c3c490d25c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee185397f66454292fd20c3c490d25c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39617 } } }
I20260812 03:12:14.526942 13819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ee185397f66454292fd20c3c490d25c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ee185397f66454292fd20c3c490d25c. Latest consensus state: current_term: 1 leader_uuid: "1ee185397f66454292fd20c3c490d25c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee185397f66454292fd20c3c490d25c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39617 } } }
I20260812 03:12:14.527077 13819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ee185397f66454292fd20c3c490d25c [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:14.527077 13818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ee185397f66454292fd20c3c490d25c [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:14.527596 13826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:12:14.531169 13826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:12:14.537577 13826 catalog_manager.cc:1347] Generated new cluster ID: 12b7f48891ab4f1286c58bad0793531e
I20260812 03:12:14.537671 13826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:12:14.560190 13826 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:12:14.561721 13826 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:12:14.571163 13826 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ee185397f66454292fd20c3c490d25c: Generated new TSK 0
I20260812 03:12:14.572103 13826 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:12:14.583732 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:0
--local_ip_for_outbound_sockets=127.26.250.129
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39617
--builtin_ntp_servers=127.26.250.148:33271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:12:14.706142 13836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:14.706511 13836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:14.706557 13836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:14.710718 13836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:14.710772 13836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:14.710829 13836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:12:14.715752 13836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:33271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:14.716913 13836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:14.718431 13836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:14.729115 13845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:14.729154 13843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:14.729115 13841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:14.729408 13836 server_base.cc:1034] running on GCE node
I20260812 03:12:14.729889 13836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:14.730712 13836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:14.731896 13836 hybrid_clock.cc:648] HybridClock initialized: now 1786504334731873 us; error 38 us; skew 500 ppm
I20260812 03:12:14.733821 13836 webserver.cc:460] Webserver started at http://127.26.250.129:38123/ using document root <none> and password file <none>
I20260812 03:12:14.734506 13836 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:14.734580 13836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:14.734771 13836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:14.736883 13836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-0/data/instance:
uuid: "01dd1b2c8c4e41cdb244b4cfcabdd9b8"
format_stamp: "Formatted at 2026-08-12 03:12:14 on dist-test-slave-zpfg"
server_key: "8c83200614f33affa9caf6c2c54b3665"
server_key_iv: "55adaec3b303fad6550693f06115d252"
server_key_version: "encryptionkey@0"
I20260812 03:12:14.737408 13836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance:
uuid: "01dd1b2c8c4e41cdb244b4cfcabdd9b8"
format_stamp: "Formatted at 2026-08-12 03:12:14 on dist-test-slave-zpfg"
server_key: "8c83200614f33affa9caf6c2c54b3665"
server_key_iv: "55adaec3b303fad6550693f06115d252"
server_key_version: "encryptionkey@0"
I20260812 03:12:14.741252 13836 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 03:12:14.744115 13851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:14.745428 13836 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:12:14.745579 13836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "01dd1b2c8c4e41cdb244b4cfcabdd9b8"
format_stamp: "Formatted at 2026-08-12 03:12:14 on dist-test-slave-zpfg"
server_key: "8c83200614f33affa9caf6c2c54b3665"
server_key_iv: "55adaec3b303fad6550693f06115d252"
server_key_version: "encryptionkey@0"
I20260812 03:12:14.745704 13836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:14.764267 13836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:14.765117 13836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:14.765334 13836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:14.765996 13836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:12:14.767051 13836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:12:14.767099 13836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:14.767169 13836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:12:14.767200 13836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:14.778687 13836 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:38405
I20260812 03:12:14.778787 13968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:38405 every 8 connection(s)
I20260812 03:12:14.780050 13836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:12:14.781915 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 13836
I20260812 03:12:14.782037 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance
I20260812 03:12:14.782439 27626 external_mini_cluster.cc:1428] Setting key a6a90a2c3ed910d583e0dce8ef611c4f
I20260812 03:12:14.784973 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:0
--local_ip_for_outbound_sockets=127.26.250.130
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39617
--builtin_ntp_servers=127.26.250.148:33271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 03:12:14.792140 13815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:14.782392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:34277 (local address 127.26.250.190:39617)
0812 03:12:14.782556 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:14.782561 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:14.783994 (+ 1433us) server_negotiation.cc:407] Connection header received
0812 03:12:14.786090 (+ 2096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:14.786093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:14.786141 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:14.786246 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:14.787559 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.788427 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.789606 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.790007 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.790145 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:14.791490 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:14.791495 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:14.791500 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:14.791520 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:14.791551 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:14.791565 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:14.791655 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:14.791818 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 03:12:14.791910 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 03:12:14.793107 13971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:14.783012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39617 (local address 127.26.250.129:34277)
0812 03:12:14.783836 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:14.783894 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:14.785822 (+ 1928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:14.786512 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:14.786524 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:14.786663 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:14.787394 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:14.787410 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.788562 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.788564 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:14.789471 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:14.789480 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.790174 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.790177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:14.790396 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:14.791152 (+ 756us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:12:14.791153 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:14.791196 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:12:14.791270 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:14.791654 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:14.791661 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:14.791665 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:12:14.791762 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 03:12:14.792013 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":169,"threads_started":1}
I20260812 03:12:14.794552 13969 heartbeater.cc:347] Connected to a master server at 127.26.250.190:39617
I20260812 03:12:14.794814 13969 heartbeater.cc:464] Registering TS with master...
I20260812 03:12:14.795418 13969 heartbeater.cc:511] Master 127.26.250.190:39617 requested a full tablet report, sending...
I20260812 03:12:14.797088 13774 ts_manager.cc:194] Registered new tserver with Master: 01dd1b2c8c4e41cdb244b4cfcabdd9b8 (127.26.250.129:38405)
I20260812 03:12:14.798300 13774 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.250.129:34277
W20260812 03:12:14.909235 13972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:14.909574 13972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:14.909617 13972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:14.913951 13972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:14.914031 13972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:14.914089 13972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:12:14.919232 13972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:33271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:14.920543 13972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:14.922081 13972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:14.933277 13980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:14.933272 13977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:14.933449 13978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:14.933699 13972 server_base.cc:1034] running on GCE node
I20260812 03:12:14.934276 13972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:14.935105 13972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:14.936305 13972 hybrid_clock.cc:648] HybridClock initialized: now 1786504334936260 us; error 54 us; skew 500 ppm
I20260812 03:12:14.938613 13972 webserver.cc:460] Webserver started at http://127.26.250.130:40223/ using document root <none> and password file <none>
I20260812 03:12:14.939369 13972 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:14.939476 13972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:14.939725 13972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:14.942085 13972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-1/data/instance:
uuid: "3115691067d344f092a3cbbc1ec73074"
format_stamp: "Formatted at 2026-08-12 03:12:14 on dist-test-slave-zpfg"
server_key: "a6b684abe212adde36c3eb8d1bf1361a"
server_key_iv: "f0a81f6b429347afe916284353ee7580"
server_key_version: "encryptionkey@0"
I20260812 03:12:14.942891 13972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance:
uuid: "3115691067d344f092a3cbbc1ec73074"
format_stamp: "Formatted at 2026-08-12 03:12:14 on dist-test-slave-zpfg"
server_key: "a6b684abe212adde36c3eb8d1bf1361a"
server_key_iv: "f0a81f6b429347afe916284353ee7580"
server_key_version: "encryptionkey@0"
I20260812 03:12:14.947233 13972 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:12:14.950078 13986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:14.951573 13972 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:12:14.951710 13972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "3115691067d344f092a3cbbc1ec73074"
format_stamp: "Formatted at 2026-08-12 03:12:14 on dist-test-slave-zpfg"
server_key: "a6b684abe212adde36c3eb8d1bf1361a"
server_key_iv: "f0a81f6b429347afe916284353ee7580"
server_key_version: "encryptionkey@0"
I20260812 03:12:14.951838 13972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:14.966065 13972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:14.966923 13972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:14.967137 13972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:14.967849 13972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:12:14.968925 13972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:12:14.968978 13972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:14.969048 13972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:12:14.969091 13972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:14.978753 13972 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:41031
I20260812 03:12:14.978776 14108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:41031 every 8 connection(s)
I20260812 03:12:14.979779 13972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:12:14.983781 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 13972
I20260812 03:12:14.983920 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance
I20260812 03:12:14.984267 27626 external_mini_cluster.cc:1428] Setting key 8c9cae81c83887f41ce9c1a731db1c30
I20260812 03:12:14.986757 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:0
--local_ip_for_outbound_sockets=127.26.250.131
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39617
--builtin_ntp_servers=127.26.250.148:33271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 03:12:14.991518 13815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:14.981693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:38797 (local address 127.26.250.190:39617)
0812 03:12:14.981855 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:14.981858 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:14.983032 (+ 1174us) server_negotiation.cc:407] Connection header received
0812 03:12:14.985251 (+ 2219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:14.985255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:14.985311 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:14.985394 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:14.986755 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.987390 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.989126 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.989521 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.989642 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:14.990876 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:14.990883 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:14.990889 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:14.990908 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:14.990936 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:14.990949 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:14.991032 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:14.991212 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 03:12:14.991291 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 03:12:14.992537 14111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:14.982128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39617 (local address 127.26.250.130:38797)
0812 03:12:14.982871 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:14.982915 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:14.985029 (+ 2114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:14.985628 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:14.985641 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:14.985764 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:14.986578 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:14.986603 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.987530 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.987536 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:14.988988 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:14.988998 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:14.989694 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:14.989697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:14.989857 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:14.990582 (+ 725us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:12:14.990584 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:14.990627 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:12:14.990697 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:14.991046 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:14.991055 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:14.991059 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 03:12:14.991163 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 03:12:14.991412 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":170,"threads_started":1}
I20260812 03:12:14.993887 14109 heartbeater.cc:347] Connected to a master server at 127.26.250.190:39617
I20260812 03:12:14.994161 14109 heartbeater.cc:464] Registering TS with master...
I20260812 03:12:14.994810 14109 heartbeater.cc:511] Master 127.26.250.190:39617 requested a full tablet report, sending...
I20260812 03:12:14.996006 13774 ts_manager.cc:194] Registered new tserver with Master: 3115691067d344f092a3cbbc1ec73074 (127.26.250.130:41031)
I20260812 03:12:14.996968 13774 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.250.130:38797
W20260812 03:12:15.112635 14112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:15.112984 14112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:15.113025 14112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:15.118145 14112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:15.118230 14112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:15.118297 14112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:12:15.123884 14112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:33271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:15.125363 14112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:15.127125 14112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:15.139040 14117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:15.139214 14112 server_base.cc:1034] running on GCE node
W20260812 03:12:15.139040 14118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:15.139045 14120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:15.139971 14112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:15.140712 14112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:15.141947 14112 hybrid_clock.cc:648] HybridClock initialized: now 1786504335141925 us; error 32 us; skew 500 ppm
I20260812 03:12:15.144403 14112 webserver.cc:460] Webserver started at http://127.26.250.131:34101/ using document root <none> and password file <none>
I20260812 03:12:15.145089 14112 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:15.145193 14112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:15.145452 14112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:15.147800 14112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-2/data/instance:
uuid: "0a1f6a44b4514a4488e73759877a27e4"
format_stamp: "Formatted at 2026-08-12 03:12:15 on dist-test-slave-zpfg"
server_key: "158f93fd9997cfb859c701574edb9b1f"
server_key_iv: "d42bf3785c493a131df9bf600b8e000e"
server_key_version: "encryptionkey@0"
I20260812 03:12:15.148424 14112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance:
uuid: "0a1f6a44b4514a4488e73759877a27e4"
format_stamp: "Formatted at 2026-08-12 03:12:15 on dist-test-slave-zpfg"
server_key: "158f93fd9997cfb859c701574edb9b1f"
server_key_iv: "d42bf3785c493a131df9bf600b8e000e"
server_key_version: "encryptionkey@0"
I20260812 03:12:15.152689 14112 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:12:15.155488 14127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:15.156901 14112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:12:15.157052 14112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "0a1f6a44b4514a4488e73759877a27e4"
format_stamp: "Formatted at 2026-08-12 03:12:15 on dist-test-slave-zpfg"
server_key: "158f93fd9997cfb859c701574edb9b1f"
server_key_iv: "d42bf3785c493a131df9bf600b8e000e"
server_key_version: "encryptionkey@0"
I20260812 03:12:15.157186 14112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:15.172902 14112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:15.173684 14112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:15.173915 14112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:15.174592 14112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:12:15.175596 14112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:12:15.175670 14112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:15.175745 14112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:12:15.175791 14112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:15.185226 14112 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:42479
I20260812 03:12:15.185261 14246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:42479 every 8 connection(s)
I20260812 03:12:15.186221 14112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:12:15.196061 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 14112
I20260812 03:12:15.196239 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance
I20260812 03:12:15.196597 27626 external_mini_cluster.cc:1428] Setting key 3fa5b9d7b3bde59273ed2b7d64f1b135
I20260812 03:12:15.196593 13815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:15.188239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:36415 (local address 127.26.250.190:39617)
0812 03:12:15.188421 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:15.188425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:15.189515 (+ 1090us) server_negotiation.cc:407] Connection header received
0812 03:12:15.190987 (+ 1472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:15.190990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:15.191041 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:15.191121 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:15.192178 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.192745 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.194121 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.194378 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.194458 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:15.195960 (+ 1502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:15.195963 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:15.195965 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:15.195978 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:15.195996 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:15.196003 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:15.196063 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:15.196341 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 03:12:15.196421 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 03:12:15.198217 14249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:15.188660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39617 (local address 127.26.250.131:36415)
0812 03:12:15.189377 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:15.189420 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:15.190778 (+ 1358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:15.191388 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:15.191396 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:15.191488 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:15.192039 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:15.192053 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.192876 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.192879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:15.194002 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:15.194014 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.194555 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.194558 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:15.194752 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:15.195585 (+ 833us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:12:15.195587 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:15.195632 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:12:15.195786 (+ 154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:15.196100 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:15.196112 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:15.196121 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 03:12:15.196805 (+ 684us) client_negotiation.cc:240] Negotiation successful
0812 03:12:15.197114 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":138,"threads_started":1}
I20260812 03:12:15.199517 14247 heartbeater.cc:347] Connected to a master server at 127.26.250.190:39617
I20260812 03:12:15.199728 14247 heartbeater.cc:464] Registering TS with master...
I20260812 03:12:15.200240 14247 heartbeater.cc:511] Master 127.26.250.190:39617 requested a full tablet report, sending...
I20260812 03:12:15.201262 13774 ts_manager.cc:194] Registered new tserver with Master: 0a1f6a44b4514a4488e73759877a27e4 (127.26.250.131:42479)
I20260812 03:12:15.201867 13774 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.250.131:36415
I20260812 03:12:15.211671 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:12:15.217989 13815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:15.213885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38834 (local address 127.26.250.190:39617)
0812 03:12:15.214020 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:15.214023 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:15.214139 (+ 116us) server_negotiation.cc:407] Connection header received
0812 03:12:15.214256 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:15.214258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:15.214380 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:15.214467 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:15.214774 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.215351 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.216465 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.216680 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.216761 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:15.217615 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:15.217618 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:15.217620 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:15.217633 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:15.217652 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:15.217660 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:15.217719 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:15.217807 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 03:12:15.217870 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 03:12:15.221239 13774 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:12:15.224592 13774 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:12:15.240553 14261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:15.235097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:41031 (local address 127.0.0.1:37150)
0812 03:12:15.235823 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:15.235839 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:15.235935 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:15.236467 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:15.236470 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:15.236488 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:15.236597 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:15.236603 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.237891 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.237895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:15.239055 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:15.239062 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.239921 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.239925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:15.240061 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:15.240139 (+ 78us) client_negotiation.cc:769] Sending connection context
0812 03:12:15.240256 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 03:12:15.240390 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":29,"thread_start_us":108,"threads_started":1}
I20260812 03:12:15.240602 14262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:15.235447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37150 (local address 127.26.250.130:41031)
0812 03:12:15.235957 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:15.235971 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:15.235999 (+ 28us) server_negotiation.cc:407] Connection header received
0812 03:12:15.236084 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:15.236090 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:15.236314 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:15.236473 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:15.236725 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.237741 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.239171 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.239789 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.239918 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:15.240106 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:15.240308 (+ 202us) server_negotiation.cc:299] Negotiation successful
0812 03:12:15.240423 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":112,"threads_started":1}
I20260812 03:12:15.241081 14258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:15.234239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:42479 (local address 127.0.0.1:55118)
0812 03:12:15.234732 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:15.234798 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:15.234955 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:15.235614 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:15.235625 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:15.235655 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:15.235833 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:15.235844 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.237400 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.237403 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:15.238884 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:15.238895 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.240040 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.240044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:15.240138 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:15.240156 (+ 18us) client_negotiation.cc:769] Sending connection context
0812 03:12:15.240233 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 03:12:15.240939 (+ 706us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":603,"thread_start_us":160,"threads_started":1}
I20260812 03:12:15.241613 14263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:15.235496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:38405 (local address 127.0.0.1:52254)
0812 03:12:15.236102 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:15.236115 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:15.236209 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:15.236854 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:15.236857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:15.236876 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:15.236966 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:15.236972 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.238417 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.238420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:15.239631 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:15.239642 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.241223 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.241227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:15.241315 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:15.241338 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:12:15.241419 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 03:12:15.241483 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":88,"thread_start_us":103,"threads_started":1}
I20260812 03:12:15.241770 14264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:15.235753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52254 (local address 127.26.250.129:38405)
0812 03:12:15.236176 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:15.236188 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:15.236209 (+ 21us) server_negotiation.cc:407] Connection header received
0812 03:12:15.236337 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:15.236349 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:15.236524 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:15.236695 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:15.237103 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.238242 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.239780 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.240570 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.240903 (+ 333us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:15.241078 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:15.241473 (+ 395us) server_negotiation.cc:299] Negotiation successful
0812 03:12:15.241583 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":103,"threads_started":1}
I20260812 03:12:15.241820 14259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:15.234473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55118 (local address 127.26.250.131:42479)
0812 03:12:15.235011 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:15.235028 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:15.235056 (+ 28us) server_negotiation.cc:407] Connection header received
0812 03:12:15.235134 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:15.235139 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:15.235408 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:15.236011 (+ 603us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:15.236200 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.237266 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.239034 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.239907 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.240408 (+ 501us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:15.240584 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:15.240641 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 03:12:15.240747 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":147,"threads_started":1}
I20260812 03:12:15.244041 14034 tablet_service.cc:1467] Processing CreateTablet for tablet 910d609cef3c467dbbcedfd8167ad5fd (DEFAULT_TABLE table=test-table [id=eab64c8d234e4cf5b5ab86841963b791]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:12:15.244673 14177 tablet_service.cc:1467] Processing CreateTablet for tablet 910d609cef3c467dbbcedfd8167ad5fd (DEFAULT_TABLE table=test-table [id=eab64c8d234e4cf5b5ab86841963b791]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:12:15.244920 13900 tablet_service.cc:1467] Processing CreateTablet for tablet 910d609cef3c467dbbcedfd8167ad5fd (DEFAULT_TABLE table=test-table [id=eab64c8d234e4cf5b5ab86841963b791]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:12:15.245420 14034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 910d609cef3c467dbbcedfd8167ad5fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:15.245828 13900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 910d609cef3c467dbbcedfd8167ad5fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:15.245834 14177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 910d609cef3c467dbbcedfd8167ad5fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:15.252645 14265 tablet_bootstrap.cc:492] T 910d609cef3c467dbbcedfd8167ad5fd P 3115691067d344f092a3cbbc1ec73074: Bootstrap starting.
I20260812 03:12:15.254400 14266 tablet_bootstrap.cc:492] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8: Bootstrap starting.
I20260812 03:12:15.254739 14267 tablet_bootstrap.cc:492] T 910d609cef3c467dbbcedfd8167ad5fd P 0a1f6a44b4514a4488e73759877a27e4: Bootstrap starting.
I20260812 03:12:15.255673 14265 tablet_bootstrap.cc:654] T 910d609cef3c467dbbcedfd8167ad5fd P 3115691067d344f092a3cbbc1ec73074: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:15.257026 14267 tablet_bootstrap.cc:654] T 910d609cef3c467dbbcedfd8167ad5fd P 0a1f6a44b4514a4488e73759877a27e4: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:15.257110 14266 tablet_bootstrap.cc:654] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:15.257287 14265 log.cc:824] T 910d609cef3c467dbbcedfd8167ad5fd P 3115691067d344f092a3cbbc1ec73074: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:15.257972 14267 log.cc:824] T 910d609cef3c467dbbcedfd8167ad5fd P 0a1f6a44b4514a4488e73759877a27e4: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:15.258136 14266 log.cc:824] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:15.260020 14265 tablet_bootstrap.cc:492] T 910d609cef3c467dbbcedfd8167ad5fd P 3115691067d344f092a3cbbc1ec73074: No bootstrap required, opened a new log
I20260812 03:12:15.260126 14266 tablet_bootstrap.cc:492] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8: No bootstrap required, opened a new log
I20260812 03:12:15.260274 14265 ts_tablet_manager.cc:1397] T 910d609cef3c467dbbcedfd8167ad5fd P 3115691067d344f092a3cbbc1ec73074: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260812 03:12:15.260309 14267 tablet_bootstrap.cc:492] T 910d609cef3c467dbbcedfd8167ad5fd P 0a1f6a44b4514a4488e73759877a27e4: No bootstrap required, opened a new log
I20260812 03:12:15.260365 14266 ts_tablet_manager.cc:1397] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260812 03:12:15.260485 14267 ts_tablet_manager.cc:1397] T 910d609cef3c467dbbcedfd8167ad5fd P 0a1f6a44b4514a4488e73759877a27e4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:12:15.265053 14266 raft_consensus.cc:348] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a1f6a44b4514a4488e73759877a27e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 42479 } } peers { permanent_uuid: "3115691067d344f092a3cbbc1ec73074" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41031 } } peers { permanent_uuid: "01dd1b2c8c4e41cdb244b4cfcabdd9b8" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 38405 } }
I20260812 03:12:15.265183 14265 raft_consensus.cc:348] T 910d609cef3c467dbbcedfd8167ad5fd P 3115691067d344f092a3cbbc1ec73074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a1f6a44b4514a4488e73759877a27e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 42479 } } peers { permanent_uuid: "3115691067d344f092a3cbbc1ec73074" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41031 } } peers { permanent_uuid: "01dd1b2c8c4e41cdb244b4cfcabdd9b8" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 38405 } }
I20260812 03:12:15.265187 14267 raft_consensus.cc:348] T 910d609cef3c467dbbcedfd8167ad5fd P 0a1f6a44b4514a4488e73759877a27e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a1f6a44b4514a4488e73759877a27e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 42479 } } peers { permanent_uuid: "3115691067d344f092a3cbbc1ec73074" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41031 } } peers { permanent_uuid: "01dd1b2c8c4e41cdb244b4cfcabdd9b8" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 38405 } }
I20260812 03:12:15.265406 14266 raft_consensus.cc:374] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:15.265468 14266 raft_consensus.cc:729] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01dd1b2c8c4e41cdb244b4cfcabdd9b8, State: Initialized, Role: FOLLOWER
I20260812 03:12:15.265484 14265 raft_consensus.cc:374] T 910d609cef3c467dbbcedfd8167ad5fd P 3115691067d344f092a3cbbc1ec73074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:15.265484 14267 raft_consensus.cc:374] T 910d609cef3c467dbbcedfd8167ad5fd P 0a1f6a44b4514a4488e73759877a27e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:15.265542 14267 raft_consensus.cc:729] T 910d609cef3c467dbbcedfd8167ad5fd P 0a1f6a44b4514a4488e73759877a27e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1f6a44b4514a4488e73759877a27e4, State: Initialized, Role: FOLLOWER
I20260812 03:12:15.265542 14265 raft_consensus.cc:729] T 910d609cef3c467dbbcedfd8167ad5fd P 3115691067d344f092a3cbbc1ec73074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3115691067d344f092a3cbbc1ec73074, State: Initialized, Role: FOLLOWER
I20260812 03:12:15.266037 14266 consensus_queue.cc:260] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a1f6a44b4514a4488e73759877a27e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 42479 } } peers { permanent_uuid: "3115691067d344f092a3cbbc1ec73074" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41031 } } peers { permanent_uuid: "01dd1b2c8c4e41cdb244b4cfcabdd9b8" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 38405 } }
I20260812 03:12:15.266053 14267 consensus_queue.cc:260] T 910d609cef3c467dbbcedfd8167ad5fd P 0a1f6a44b4514a4488e73759877a27e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a1f6a44b4514a4488e73759877a27e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 42479 } } peers { permanent_uuid: "3115691067d344f092a3cbbc1ec73074" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41031 } } peers { permanent_uuid: "01dd1b2c8c4e41cdb244b4cfcabdd9b8" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 38405 } }
I20260812 03:12:15.266239 14265 consensus_queue.cc:260] T 910d609cef3c467dbbcedfd8167ad5fd P 3115691067d344f092a3cbbc1ec73074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a1f6a44b4514a4488e73759877a27e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 42479 } } peers { permanent_uuid: "3115691067d344f092a3cbbc1ec73074" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41031 } } peers { permanent_uuid: "01dd1b2c8c4e41cdb244b4cfcabdd9b8" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 38405 } }
I20260812 03:12:15.267118 14247 heartbeater.cc:503] Master 127.26.250.190:39617 was elected leader, sending a full tablet report...
I20260812 03:12:15.267161 13969 heartbeater.cc:503] Master 127.26.250.190:39617 was elected leader, sending a full tablet report...
I20260812 03:12:15.267329 14265 ts_tablet_manager.cc:1428] T 910d609cef3c467dbbcedfd8167ad5fd P 3115691067d344f092a3cbbc1ec73074: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 03:12:15.267339 14266 ts_tablet_manager.cc:1428] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260812 03:12:15.267663 14109 heartbeater.cc:503] Master 127.26.250.190:39617 was elected leader, sending a full tablet report...
I20260812 03:12:15.269825 14267 ts_tablet_manager.cc:1428] T 910d609cef3c467dbbcedfd8167ad5fd P 0a1f6a44b4514a4488e73759877a27e4: Time spent starting tablet: real 0.009s user 0.004s sys 0.004s
W20260812 03:12:15.281443 13970 tablet.cc:2369] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:12:15.437340 14248 tablet.cc:2369] T 910d609cef3c467dbbcedfd8167ad5fd P 0a1f6a44b4514a4488e73759877a27e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:12:15.481099 14110 tablet.cc:2369] T 910d609cef3c467dbbcedfd8167ad5fd P 3115691067d344f092a3cbbc1ec73074: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:12:15.487244 14271 raft_consensus.cc:482] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:12:15.487550 14271 raft_consensus.cc:504] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a1f6a44b4514a4488e73759877a27e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 42479 } } peers { permanent_uuid: "3115691067d344f092a3cbbc1ec73074" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41031 } } peers { permanent_uuid: "01dd1b2c8c4e41cdb244b4cfcabdd9b8" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 38405 } }
I20260812 03:12:15.488767 14271 leader_election.cc:288] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a1f6a44b4514a4488e73759877a27e4 (127.26.250.131:42479), 3115691067d344f092a3cbbc1ec73074 (127.26.250.130:41031)
I20260812 03:12:15.493583 14259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:15.489229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:39803 (local address 127.26.250.131:42479)
0812 03:12:15.489407 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:15.489411 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:15.489454 (+ 43us) server_negotiation.cc:407] Connection header received
0812 03:12:15.489649 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:15.489653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:15.489707 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:15.489806 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:15.490155 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.490880 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.492384 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.492951 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.493097 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:15.493171 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:15.493398 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 03:12:15.493463 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:12:15.493657 14274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:15.489088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:42479 (local address 127.26.250.129:39803)
0812 03:12:15.489375 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:15.489393 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:15.489515 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:15.489842 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:15.489845 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:15.489867 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:15.490004 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:15.490009 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.491034 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.491038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:15.492237 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:15.492248 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.493128 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.493133 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:15.493248 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:15.493280 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 03:12:15.493375 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 03:12:15.493457 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":113,"threads_started":1}
I20260812 03:12:15.494211 14262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:15.489690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:37973 (local address 127.26.250.130:41031)
0812 03:12:15.489865 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:15.489869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:15.490085 (+ 216us) server_negotiation.cc:407] Connection header received
0812 03:12:15.490202 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:15.490204 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:15.490247 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:15.490312 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:15.490718 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.491615 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.492818 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.493570 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.493716 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:15.493781 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:15.493998 (+ 217us) server_negotiation.cc:299] Negotiation successful
0812 03:12:15.494088 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 03:12:15.494635 14275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:15.489521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:41031 (local address 127.26.250.129:37973)
0812 03:12:15.489981 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:15.489992 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:15.490100 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:15.490369 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:15.490372 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:15.490392 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:15.490514 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:15.490521 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.491751 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.491754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:15.492676 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:15.492686 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.493710 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.493714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:15.493843 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:15.493870 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 03:12:15.494413 (+ 543us) client_negotiation.cc:240] Negotiation successful
0812 03:12:15.494501 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"mutex_wait_us":1,"thread_start_us":122,"threads_started":1}
I20260812 03:12:15.494735 14199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "910d609cef3c467dbbcedfd8167ad5fd" candidate_uuid: "01dd1b2c8c4e41cdb244b4cfcabdd9b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1f6a44b4514a4488e73759877a27e4" is_pre_election: true
I20260812 03:12:15.495113 14199 raft_consensus.cc:2454] T 910d609cef3c467dbbcedfd8167ad5fd P 0a1f6a44b4514a4488e73759877a27e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01dd1b2c8c4e41cdb244b4cfcabdd9b8 in term 0.
I20260812 03:12:15.495244 14059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "910d609cef3c467dbbcedfd8167ad5fd" candidate_uuid: "01dd1b2c8c4e41cdb244b4cfcabdd9b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3115691067d344f092a3cbbc1ec73074" is_pre_election: true
I20260812 03:12:15.495556 14059 raft_consensus.cc:2454] T 910d609cef3c467dbbcedfd8167ad5fd P 3115691067d344f092a3cbbc1ec73074 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01dd1b2c8c4e41cdb244b4cfcabdd9b8 in term 0.
I20260812 03:12:15.495728 13856 leader_election.cc:302] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01dd1b2c8c4e41cdb244b4cfcabdd9b8, 0a1f6a44b4514a4488e73759877a27e4; no voters:
I20260812 03:12:15.496026 14271 raft_consensus.cc:2781] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:12:15.496151 14271 raft_consensus.cc:482] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:12:15.496217 14271 raft_consensus.cc:3037] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:15.497329 14271 raft_consensus.cc:504] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a1f6a44b4514a4488e73759877a27e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 42479 } } peers { permanent_uuid: "3115691067d344f092a3cbbc1ec73074" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41031 } } peers { permanent_uuid: "01dd1b2c8c4e41cdb244b4cfcabdd9b8" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 38405 } }
I20260812 03:12:15.497718 14271 leader_election.cc:288] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8 [CANDIDATE]: Term 1 election: Requested vote from peers 0a1f6a44b4514a4488e73759877a27e4 (127.26.250.131:42479), 3115691067d344f092a3cbbc1ec73074 (127.26.250.130:41031)
I20260812 03:12:15.498196 14059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "910d609cef3c467dbbcedfd8167ad5fd" candidate_uuid: "01dd1b2c8c4e41cdb244b4cfcabdd9b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3115691067d344f092a3cbbc1ec73074"
I20260812 03:12:15.498196 14199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "910d609cef3c467dbbcedfd8167ad5fd" candidate_uuid: "01dd1b2c8c4e41cdb244b4cfcabdd9b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1f6a44b4514a4488e73759877a27e4"
I20260812 03:12:15.498368 14059 raft_consensus.cc:3037] T 910d609cef3c467dbbcedfd8167ad5fd P 3115691067d344f092a3cbbc1ec73074 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:15.498335 14199 raft_consensus.cc:3037] T 910d609cef3c467dbbcedfd8167ad5fd P 0a1f6a44b4514a4488e73759877a27e4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:15.499820 14059 raft_consensus.cc:2454] T 910d609cef3c467dbbcedfd8167ad5fd P 3115691067d344f092a3cbbc1ec73074 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01dd1b2c8c4e41cdb244b4cfcabdd9b8 in term 1.
I20260812 03:12:15.499828 14199 raft_consensus.cc:2454] T 910d609cef3c467dbbcedfd8167ad5fd P 0a1f6a44b4514a4488e73759877a27e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01dd1b2c8c4e41cdb244b4cfcabdd9b8 in term 1.
I20260812 03:12:15.500161 13856 leader_election.cc:302] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01dd1b2c8c4e41cdb244b4cfcabdd9b8, 3115691067d344f092a3cbbc1ec73074; no voters:
I20260812 03:12:15.500370 14271 raft_consensus.cc:2781] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:15.500629 14271 raft_consensus.cc:686] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8 [term 1 LEADER]: Becoming Leader. State: Replica: 01dd1b2c8c4e41cdb244b4cfcabdd9b8, State: Running, Role: LEADER
I20260812 03:12:15.501012 14271 consensus_queue.cc:237] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a1f6a44b4514a4488e73759877a27e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 42479 } } peers { permanent_uuid: "3115691067d344f092a3cbbc1ec73074" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41031 } } peers { permanent_uuid: "01dd1b2c8c4e41cdb244b4cfcabdd9b8" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 38405 } }
I20260812 03:12:15.504276 13771 catalog_manager.cc:5495] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01dd1b2c8c4e41cdb244b4cfcabdd9b8 (127.26.250.129). New cstate: current_term: 1 leader_uuid: "01dd1b2c8c4e41cdb244b4cfcabdd9b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a1f6a44b4514a4488e73759877a27e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 42479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3115691067d344f092a3cbbc1ec73074" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01dd1b2c8c4e41cdb244b4cfcabdd9b8" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 38405 } health_report { overall_health: HEALTHY } } }
I20260812 03:12:15.537256 14264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:15.533086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52256 (local address 127.26.250.129:38405)
0812 03:12:15.533235 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:15.533240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:15.533270 (+ 30us) server_negotiation.cc:407] Connection header received
0812 03:12:15.533507 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:15.533510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:15.533570 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:15.533792 (+ 222us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:12:15.534052 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.534689 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.535660 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:15.535890 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:15.536008 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:15.536381 (+ 373us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:12:15.536833 (+ 452us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:12:15.536987 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:15.537030 (+ 43us) server_negotiation.cc:299] Negotiation successful
0812 03:12:15.537098 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 03:12:15.543694 14059 raft_consensus.cc:1261] T 910d609cef3c467dbbcedfd8167ad5fd P 3115691067d344f092a3cbbc1ec73074 [term 1 FOLLOWER]: Refusing update from remote peer 01dd1b2c8c4e41cdb244b4cfcabdd9b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:12:15.543752 14199 raft_consensus.cc:1261] T 910d609cef3c467dbbcedfd8167ad5fd P 0a1f6a44b4514a4488e73759877a27e4 [term 1 FOLLOWER]: Refusing update from remote peer 01dd1b2c8c4e41cdb244b4cfcabdd9b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:12:15.544502 14276 consensus_queue.cc:1035] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3115691067d344f092a3cbbc1ec73074" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:12:15.544711 14271 consensus_queue.cc:1035] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1f6a44b4514a4488e73759877a27e4" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 42479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:12:15.555235 14282 mvcc.cc:205] Tried to move back new op lower bound from 7317521758378291200 to 7317521758214959104. Current Snapshot: MvccSnapshot[applied={T|T < 7317521758378291200}]
I20260812 03:12:15.561760 13775 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:12:15.561952 13775 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:12:15.564545 13775 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1ee185397f66454292fd20c3c490d25c: Sending DeleteTablet for 3 replicas of tablet 910d609cef3c467dbbcedfd8167ad5fd
I20260812 03:12:15.565428 14177 tablet_service.cc:1514] Processing DeleteTablet for tablet 910d609cef3c467dbbcedfd8167ad5fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:12:15 UTC) from {username='slave'} at 127.0.0.1:55118
I20260812 03:12:15.565636 14034 tablet_service.cc:1514] Processing DeleteTablet for tablet 910d609cef3c467dbbcedfd8167ad5fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:12:15 UTC) from {username='slave'} at 127.0.0.1:37150
I20260812 03:12:15.565627 13900 tablet_service.cc:1514] Processing DeleteTablet for tablet 910d609cef3c467dbbcedfd8167ad5fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:12:15 UTC) from {username='slave'} at 127.0.0.1:52254
I20260812 03:12:15.566002 14290 tablet_replica.cc:330] T 910d609cef3c467dbbcedfd8167ad5fd P 0a1f6a44b4514a4488e73759877a27e4: stopping tablet replica
I20260812 03:12:15.566169 14292 tablet_replica.cc:330] T 910d609cef3c467dbbcedfd8167ad5fd P 3115691067d344f092a3cbbc1ec73074: stopping tablet replica
I20260812 03:12:15.566509 14292 raft_consensus.cc:2229] T 910d609cef3c467dbbcedfd8167ad5fd P 3115691067d344f092a3cbbc1ec73074 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:12:15.566504 14290 raft_consensus.cc:2229] T 910d609cef3c467dbbcedfd8167ad5fd P 0a1f6a44b4514a4488e73759877a27e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:12:15.566884 14290 raft_consensus.cc:2258] T 910d609cef3c467dbbcedfd8167ad5fd P 0a1f6a44b4514a4488e73759877a27e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:12:15.567126 14293 tablet_replica.cc:330] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8: stopping tablet replica
I20260812 03:12:15.567178 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 13836
I20260812 03:12:15.567425 14293 raft_consensus.cc:2229] T 910d609cef3c467dbbcedfd8167ad5fd P 01dd1b2c8c4e41cdb244b4cfcabdd9b8 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:12:15.571247 14292 raft_consensus.cc:2258] T 910d609cef3c467dbbcedfd8167ad5fd P 3115691067d344f092a3cbbc1ec73074 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:12:15.572765 14290 ts_tablet_manager.cc:1905] T 910d609cef3c467dbbcedfd8167ad5fd P 0a1f6a44b4514a4488e73759877a27e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:12:15.572765 14292 ts_tablet_manager.cc:1905] T 910d609cef3c467dbbcedfd8167ad5fd P 3115691067d344f092a3cbbc1ec73074: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:12:15.575793 14292 ts_tablet_manager.cc:1918] T 910d609cef3c467dbbcedfd8167ad5fd P 3115691067d344f092a3cbbc1ec73074: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:12:15.575793 14290 ts_tablet_manager.cc:1918] T 910d609cef3c467dbbcedfd8167ad5fd P 0a1f6a44b4514a4488e73759877a27e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:12:15.575922 14292 log.cc:1196] T 910d609cef3c467dbbcedfd8167ad5fd P 3115691067d344f092a3cbbc1ec73074: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-1/wal/wals/910d609cef3c467dbbcedfd8167ad5fd
I20260812 03:12:15.575922 14290 log.cc:1196] T 910d609cef3c467dbbcedfd8167ad5fd P 0a1f6a44b4514a4488e73759877a27e4: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/910d609cef3c467dbbcedfd8167ad5fd
I20260812 03:12:15.576289 14292 ts_tablet_manager.cc:1939] T 910d609cef3c467dbbcedfd8167ad5fd P 3115691067d344f092a3cbbc1ec73074: Deleting consensus metadata
I20260812 03:12:15.576308 14290 ts_tablet_manager.cc:1939] T 910d609cef3c467dbbcedfd8167ad5fd P 0a1f6a44b4514a4488e73759877a27e4: Deleting consensus metadata
I20260812 03:12:15.577880 13762 catalog_manager.cc:4841] TS 3115691067d344f092a3cbbc1ec73074 (127.26.250.130:41031): tablet 910d609cef3c467dbbcedfd8167ad5fd (table test-table [id=eab64c8d234e4cf5b5ab86841963b791]) successfully deleted
I20260812 03:12:15.578137 13762 catalog_manager.cc:4841] TS 0a1f6a44b4514a4488e73759877a27e4 (127.26.250.131:42479): tablet 910d609cef3c467dbbcedfd8167ad5fd (table test-table [id=eab64c8d234e4cf5b5ab86841963b791]) successfully deleted
W20260812 03:12:15.578464 13762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.250.129:38405 (error 108)
I20260812 03:12:15.578642 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 13972
I20260812 03:12:15.580442 14263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:15.580141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:38405 (local address 127.0.0.1:52264)
0812 03:12:15.580283 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:15.580366 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.129:38405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 03:12:15.580607 13762 catalog_manager.cc:4584] TS 01dd1b2c8c4e41cdb244b4cfcabdd9b8 (127.26.250.129:38405): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 910d609cef3c467dbbcedfd8167ad5fd: Network error: Client connection negotiation failed: client connection to 127.26.250.129:38405: connect: Connection refused (error 111)
I20260812 03:12:15.588631 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 14112
I20260812 03:12:15.597836 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 13743
2026-08-12T03:12:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1326 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:12:15 dist-test-slave-zpfg krb5kdc[14297](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:12:15 dist-test-slave-zpfg krb5kdc[14297](info): set up 2 sockets
Aug 12 03:12:15 dist-test-slave-zpfg krb5kdc[14297](info): commencing operation
krb5kdc: starting...
W20260812 03:12:16.688037 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:12:16 dist-test-slave-zpfg krb5kdc[14297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:12:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:12:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:12:16.830330 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:38809
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:34205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:38809
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:16.955065 14310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:16.955401 14310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:16.955479 14310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:16.959681 14310 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:16.959769 14310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:16.959808 14310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:16.959867 14310 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:16.959906 14310 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:16.965373 14310 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:34205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:38809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:38809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:16.966527 14310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:16.967623 14310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:16.976643 14319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:16.976758 14310 server_base.cc:1034] running on GCE node
W20260812 03:12:16.976629 14317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:16.976634 14316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:16.977561 14310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:16.978859 14310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:16.980089 14310 hybrid_clock.cc:648] HybridClock initialized: now 1786504336980060 us; error 43 us; skew 500 ppm
Aug 12 03:12:16 dist-test-slave-zpfg krb5kdc[14297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504336, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:16.983505 14310 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:12:16.984555 14310 webserver.cc:460] Webserver started at http://127.26.250.190:35725/ using document root <none> and password file <none>
I20260812 03:12:16.985172 14310 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:16.985277 14310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:16.985528 14310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:16.987875 14310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "75de098b2947496bac7042b71ea2e8b0"
format_stamp: "Formatted at 2026-08-12 03:12:16 on dist-test-slave-zpfg"
server_key: "e0ed70bfc19034e07271914eb548a9b8"
server_key_iv: "ea9110a11e3a0b737285140b5e3bbf73"
server_key_version: "encryptionkey@0"
I20260812 03:12:16.988585 14310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "75de098b2947496bac7042b71ea2e8b0"
format_stamp: "Formatted at 2026-08-12 03:12:16 on dist-test-slave-zpfg"
server_key: "e0ed70bfc19034e07271914eb548a9b8"
server_key_iv: "ea9110a11e3a0b737285140b5e3bbf73"
server_key_version: "encryptionkey@0"
I20260812 03:12:16.992831 14310 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 03:12:16.995970 14326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:16.997493 14310 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:12:16.997756 14310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "75de098b2947496bac7042b71ea2e8b0"
format_stamp: "Formatted at 2026-08-12 03:12:16 on dist-test-slave-zpfg"
server_key: "e0ed70bfc19034e07271914eb548a9b8"
server_key_iv: "ea9110a11e3a0b737285140b5e3bbf73"
server_key_version: "encryptionkey@0"
I20260812 03:12:16.997895 14310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:17.015769 14310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:17.019172 14310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:17.019402 14310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:17.027522 14310 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:38809
I20260812 03:12:17.027547 14381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:38809 every 8 connection(s)
I20260812 03:12:17.028680 14310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:12:17.032027 14382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:17.038743 14382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75de098b2947496bac7042b71ea2e8b0: Bootstrap starting.
I20260812 03:12:17.038959 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 14310
I20260812 03:12:17.039117 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:12:17.039419 27626 external_mini_cluster.cc:1428] Setting key cac75a95ebba1eca585bbb649f628392
I20260812 03:12:17.041584 14382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75de098b2947496bac7042b71ea2e8b0: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:17.042498 14382 log.cc:824] T 00000000000000000000000000000000 P 75de098b2947496bac7042b71ea2e8b0: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:17.044657 14382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75de098b2947496bac7042b71ea2e8b0: No bootstrap required, opened a new log
I20260812 03:12:17.048635 14382 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75de098b2947496bac7042b71ea2e8b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75de098b2947496bac7042b71ea2e8b0" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 38809 } }
Aug 12 03:12:17 dist-test-slave-zpfg krb5kdc[14297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:12:17.048970 14382 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75de098b2947496bac7042b71ea2e8b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:17.049031 14382 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75de098b2947496bac7042b71ea2e8b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75de098b2947496bac7042b71ea2e8b0, State: Initialized, Role: FOLLOWER
I20260812 03:12:17.049592 14382 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75de098b2947496bac7042b71ea2e8b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75de098b2947496bac7042b71ea2e8b0" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 38809 } }
I20260812 03:12:17.049755 14382 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75de098b2947496bac7042b71ea2e8b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:12:17.049871 14382 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75de098b2947496bac7042b71ea2e8b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:12:17.050000 14382 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75de098b2947496bac7042b71ea2e8b0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:17.051368 14382 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75de098b2947496bac7042b71ea2e8b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75de098b2947496bac7042b71ea2e8b0" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 38809 } }
I20260812 03:12:17.051752 14382 leader_election.cc:302] T 00000000000000000000000000000000 P 75de098b2947496bac7042b71ea2e8b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75de098b2947496bac7042b71ea2e8b0; no voters:
I20260812 03:12:17.052161 14382 leader_election.cc:288] T 00000000000000000000000000000000 P 75de098b2947496bac7042b71ea2e8b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:12:17.052282 14388 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75de098b2947496bac7042b71ea2e8b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:17.052547 14388 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75de098b2947496bac7042b71ea2e8b0 [term 1 LEADER]: Becoming Leader. State: Replica: 75de098b2947496bac7042b71ea2e8b0, State: Running, Role: LEADER
I20260812 03:12:17.052904 14388 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75de098b2947496bac7042b71ea2e8b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75de098b2947496bac7042b71ea2e8b0" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 38809 } }
I20260812 03:12:17.053582 14382 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75de098b2947496bac7042b71ea2e8b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:17.054221 14388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75de098b2947496bac7042b71ea2e8b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75de098b2947496bac7042b71ea2e8b0. Latest consensus state: current_term: 1 leader_uuid: "75de098b2947496bac7042b71ea2e8b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75de098b2947496bac7042b71ea2e8b0" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 38809 } } }
I20260812 03:12:17.054396 14388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75de098b2947496bac7042b71ea2e8b0 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:17.054796 14396 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:12:17.054746 14392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75de098b2947496bac7042b71ea2e8b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75de098b2947496bac7042b71ea2e8b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75de098b2947496bac7042b71ea2e8b0" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 38809 } } }
I20260812 03:12:17.054863 14392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75de098b2947496bac7042b71ea2e8b0 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:17.056195 14385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:17.040929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57532 (local address 127.26.250.190:38809)
0812 03:12:17.041356 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:17.041381 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:17.041418 (+ 37us) server_negotiation.cc:407] Connection header received
0812 03:12:17.042214 (+ 796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:17.042235 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:17.042720 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:17.043144 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:17.044013 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.045075 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.045980 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.046535 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.046708 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:17.049653 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:17.049658 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:17.049672 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:17.049702 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:17.051950 (+ 2248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:17.052506 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:17.052515 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:17.052524 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:17.052711 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:17.053108 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:17.053111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:17.053113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:17.053627 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:17.053802 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:17.054779 (+ 977us) server_negotiation.cc:299] Negotiation successful
0812 03:12:17.055127 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":147,"threads_started":1}
I20260812 03:12:17.060418 14396 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:12:17.067121 14396 catalog_manager.cc:1347] Generated new cluster ID: 48d3a94d2b434b5bace00fa69419700b
I20260812 03:12:17.067313 14396 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:12:17.076403 14396 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:12:17.077519 14396 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:12:17.086010 14396 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75de098b2947496bac7042b71ea2e8b0: Generated new TSK 0
I20260812 03:12:17.086912 14396 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.250.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
I20260812 03:12:17.125921 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:0
--local_ip_for_outbound_sockets=127.26.250.129
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:38809
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:34205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:17.276185 14411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:17.276499 14411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:17.276546 14411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:17.280848 14411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:17.280939 14411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:17.281003 14411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:12:17.286708 14411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:34205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:38809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:17.287844 14411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:17.289000 14411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:17.299957 14418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:17.299968 14417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:17.299959 14420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:17.300578 14411 server_base.cc:1034] running on GCE node
I20260812 03:12:17.301088 14411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:17.301877 14411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:17.303099 14411 hybrid_clock.cc:648] HybridClock initialized: now 1786504337303056 us; error 55 us; skew 500 ppm
Aug 12 03:12:17 dist-test-slave-zpfg krb5kdc[14297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504337, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:17.306617 14411 init.cc:377] Logged in from keytab as kudu/127.26.250.129@KRBTEST.COM (short username kudu)
I20260812 03:12:17.307694 14411 webserver.cc:460] Webserver started at http://127.26.250.129:38033/ using document root <none> and password file <none>
I20260812 03:12:17.308327 14411 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:17.308429 14411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:17.308702 14411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:17.311090 14411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-0/data/instance:
uuid: "538fe07ab0fd4032837ffe52d4e0f1b5"
format_stamp: "Formatted at 2026-08-12 03:12:17 on dist-test-slave-zpfg"
server_key: "c5982f9ac3eebd891070146fa3598cf1"
server_key_iv: "47711eac19505d12334a0155b72ee860"
server_key_version: "encryptionkey@0"
I20260812 03:12:17.311755 14411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance:
uuid: "538fe07ab0fd4032837ffe52d4e0f1b5"
format_stamp: "Formatted at 2026-08-12 03:12:17 on dist-test-slave-zpfg"
server_key: "c5982f9ac3eebd891070146fa3598cf1"
server_key_iv: "47711eac19505d12334a0155b72ee860"
server_key_version: "encryptionkey@0"
I20260812 03:12:17.316282 14411 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:12:17.319682 14428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:17.321444 14411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:12:17.321637 14411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "538fe07ab0fd4032837ffe52d4e0f1b5"
format_stamp: "Formatted at 2026-08-12 03:12:17 on dist-test-slave-zpfg"
server_key: "c5982f9ac3eebd891070146fa3598cf1"
server_key_iv: "47711eac19505d12334a0155b72ee860"
server_key_version: "encryptionkey@0"
I20260812 03:12:17.321795 14411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:17.342041 14411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:17.346323 14411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:17.346637 14411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:17.347360 14411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:12:17.348445 14411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:12:17.348531 14411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:17.348603 14411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:12:17.348655 14411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:17.359516 14411 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:32999
I20260812 03:12:17.359536 14555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:32999 every 8 connection(s)
I20260812 03:12:17.360488 14411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:12:17.366641 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 14411
I20260812 03:12:17.366870 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance
I20260812 03:12:17.367239 27626 external_mini_cluster.cc:1428] Setting key efb205b0e9c497a33a5a3e458973a6db
Aug 12 03:12:17 dist-test-slave-zpfg krb5kdc[14297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504337, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:12:17.376330 14385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:17.362508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:37641 (local address 127.26.250.190:38809)
0812 03:12:17.362681 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:17.362685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:17.363652 (+ 967us) server_negotiation.cc:407] Connection header received
0812 03:12:17.365156 (+ 1504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:17.365159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:17.365222 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:17.365303 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:17.366803 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.367606 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.369147 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.369604 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.369734 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:17.372478 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:17.372489 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:17.372491 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:17.372516 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:17.374052 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:17.374709 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:17.374712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:17.374713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:17.374792 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:17.375169 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:17.375172 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:17.375174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:17.375369 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:17.375477 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:17.376011 (+ 534us) server_negotiation.cc:299] Negotiation successful
0812 03:12:17.376159 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 03:12:17.377228 14558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:17.362961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:38809 (local address 127.26.250.129:37641)
0812 03:12:17.363475 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:17.363526 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:17.364919 (+ 1393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:17.365491 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:17.365499 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:17.365943 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:17.366612 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:17.366630 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.367779 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.367783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:17.368999 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:17.369010 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.369765 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.369768 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:17.369944 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:17.370691 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:17.370709 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:17.372284 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:17.374185 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:17.374191 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:17.374209 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:17.374591 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:17.374899 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:17.374902 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:17.374904 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:17.375064 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:17.375477 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:17.375485 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:17.375767 (+ 282us) client_negotiation.cc:769] Sending connection context
0812 03:12:17.375956 (+ 189us) client_negotiation.cc:240] Negotiation successful
0812 03:12:17.376347 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":123,"threads_started":1}
WARNING: no policy specified for kudu/127.26.250.130@KRBTEST.COM; defaulting to no policy
I20260812 03:12:17.378628 14556 heartbeater.cc:347] Connected to a master server at 127.26.250.190:38809
I20260812 03:12:17.378916 14556 heartbeater.cc:464] Registering TS with master...
I20260812 03:12:17.379496 14556 heartbeater.cc:511] Master 127.26.250.190:38809 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.130@KRBTEST.COM" created.
I20260812 03:12:17.381455 14344 ts_manager.cc:194] Registered new tserver with Master: 538fe07ab0fd4032837ffe52d4e0f1b5 (127.26.250.129:32999)
I20260812 03:12:17.382732 14344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.129@KRBTEST.COM'} at 127.26.250.129:37641
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
I20260812 03:12:17.397684 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:0
--local_ip_for_outbound_sockets=127.26.250.130
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:38809
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:34205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:17.520077 14561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:17.520380 14561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:17.520427 14561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:17.524734 14561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:17.524814 14561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:17.524928 14561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:12:17.530687 14561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:34205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:38809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:17.531881 14561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:17.533043 14561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:12:17.543139 14561 server_base.cc:1034] running on GCE node
W20260812 03:12:17.543112 14566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:17.543175 14568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:17.543339 14570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:17.543824 14561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:17.544636 14561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:17.545814 14561 hybrid_clock.cc:648] HybridClock initialized: now 1786504337545786 us; error 40 us; skew 500 ppm
Aug 12 03:12:17 dist-test-slave-zpfg krb5kdc[14297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504337, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:17.548799 14561 init.cc:377] Logged in from keytab as kudu/127.26.250.130@KRBTEST.COM (short username kudu)
I20260812 03:12:17.549695 14561 webserver.cc:460] Webserver started at http://127.26.250.130:40401/ using document root <none> and password file <none>
I20260812 03:12:17.550292 14561 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:17.550414 14561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:17.550608 14561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:17.552671 14561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-1/data/instance:
uuid: "d622e984b9b74ec3ac213f1d137c3b83"
format_stamp: "Formatted at 2026-08-12 03:12:17 on dist-test-slave-zpfg"
server_key: "d7a6756b1800979702082322fcfe3e24"
server_key_iv: "853f3c43235809ab064ffe924fff0975"
server_key_version: "encryptionkey@0"
I20260812 03:12:17.553205 14561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance:
uuid: "d622e984b9b74ec3ac213f1d137c3b83"
format_stamp: "Formatted at 2026-08-12 03:12:17 on dist-test-slave-zpfg"
server_key: "d7a6756b1800979702082322fcfe3e24"
server_key_iv: "853f3c43235809ab064ffe924fff0975"
server_key_version: "encryptionkey@0"
I20260812 03:12:17.557436 14561 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260812 03:12:17.560072 14577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:17.561455 14561 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260812 03:12:17.561555 14561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "d622e984b9b74ec3ac213f1d137c3b83"
format_stamp: "Formatted at 2026-08-12 03:12:17 on dist-test-slave-zpfg"
server_key: "d7a6756b1800979702082322fcfe3e24"
server_key_iv: "853f3c43235809ab064ffe924fff0975"
server_key_version: "encryptionkey@0"
I20260812 03:12:17.561648 14561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:17.596110 14561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:17.599826 14561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:17.600049 14561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:17.600703 14561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:12:17.601647 14561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:12:17.601715 14561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:17.601783 14561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:12:17.601830 14561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:17.611661 14561 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:33115
I20260812 03:12:17.611732 14701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:33115 every 8 connection(s)
I20260812 03:12:17.612704 14561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:12:17.616771 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 14561
I20260812 03:12:17.616945 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance
I20260812 03:12:17.617235 27626 external_mini_cluster.cc:1428] Setting key fd8c5f41322abdbd28220908d6d4140e
Aug 12 03:12:17 dist-test-slave-zpfg krb5kdc[14297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504337, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.250.131@KRBTEST.COM; defaulting to no policy
I20260812 03:12:17.629467 14385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:17.614802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:48461 (local address 127.26.250.190:38809)
0812 03:12:17.615080 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:17.615084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:17.615917 (+ 833us) server_negotiation.cc:407] Connection header received
0812 03:12:17.617472 (+ 1555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:17.617476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:17.617534 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:17.617636 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:17.619127 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.619973 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.621413 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.621633 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.621716 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:17.625036 (+ 3320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:17.625044 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:17.625050 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:17.625089 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:17.627033 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:17.627700 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:17.627707 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:17.627710 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:17.627809 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:17.628166 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:17.628170 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:17.628172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:17.628449 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:17.628563 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:17.629104 (+ 541us) server_negotiation.cc:299] Negotiation successful
0812 03:12:17.629239 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
I20260812 03:12:17.630393 14704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:17.615228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:38809 (local address 127.26.250.130:48461)
0812 03:12:17.615713 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:17.615760 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:17.617223 (+ 1463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:17.617900 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:17.617907 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:17.618271 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:17.618963 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:17.618980 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.620125 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.620128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:17.621268 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:17.621282 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.621801 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.621808 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:17.622081 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:17.622960 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:17.622988 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:17.624835 (+ 1847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:17.627233 (+ 2398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:17.627241 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:17.627264 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:17.627582 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:17.627914 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:17.627916 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:17.627918 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:17.628060 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:17.628565 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:17.628570 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:17.628846 (+ 276us) client_negotiation.cc:769] Sending connection context
0812 03:12:17.629013 (+ 167us) client_negotiation.cc:240] Negotiation successful
0812 03:12:17.629347 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":109,"threads_started":1}
I20260812 03:12:17.631707 14702 heartbeater.cc:347] Connected to a master server at 127.26.250.190:38809
I20260812 03:12:17.631980 14702 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.131@KRBTEST.COM" created.
I20260812 03:12:17.632659 14702 heartbeater.cc:511] Master 127.26.250.190:38809 requested a full tablet report, sending...
I20260812 03:12:17.633812 14344 ts_manager.cc:194] Registered new tserver with Master: d622e984b9b74ec3ac213f1d137c3b83 (127.26.250.130:33115)
I20260812 03:12:17.634506 14344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.130@KRBTEST.COM'} at 127.26.250.130:48461
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
I20260812 03:12:17.650110 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:0
--local_ip_for_outbound_sockets=127.26.250.131
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:38809
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:34205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:17.768491 14707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:17.768777 14707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:17.768822 14707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:17.772965 14707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:17.773036 14707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:17.773128 14707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:12:17.782893 14707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:34205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:38809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:17.784457 14707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:17.785794 14707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:17.798425 14716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:17.798643 14713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:17.798434 14707 server_base.cc:1034] running on GCE node
W20260812 03:12:17.801560 14714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:17.802124 14707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:17.803067 14707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:17.804261 14707 hybrid_clock.cc:648] HybridClock initialized: now 1786504337804183 us; error 98 us; skew 500 ppm
Aug 12 03:12:17 dist-test-slave-zpfg krb5kdc[14297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504337, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:17.807287 14707 init.cc:377] Logged in from keytab as kudu/127.26.250.131@KRBTEST.COM (short username kudu)
I20260812 03:12:17.808241 14707 webserver.cc:460] Webserver started at http://127.26.250.131:36779/ using document root <none> and password file <none>
I20260812 03:12:17.808867 14707 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:17.808970 14707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:17.809218 14707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:17.811484 14707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-2/data/instance:
uuid: "3e0da1f0c2584f36a1bca4f01e8f6ae0"
format_stamp: "Formatted at 2026-08-12 03:12:17 on dist-test-slave-zpfg"
server_key: "53861a365b3d2e3316d8021c960af9ca"
server_key_iv: "0f9f5ae654cb16d79cc79c8923e24723"
server_key_version: "encryptionkey@0"
I20260812 03:12:17.812121 14707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance:
uuid: "3e0da1f0c2584f36a1bca4f01e8f6ae0"
format_stamp: "Formatted at 2026-08-12 03:12:17 on dist-test-slave-zpfg"
server_key: "53861a365b3d2e3316d8021c960af9ca"
server_key_iv: "0f9f5ae654cb16d79cc79c8923e24723"
server_key_version: "encryptionkey@0"
I20260812 03:12:17.816591 14707 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:12:17.819660 14724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:17.821066 14707 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 03:12:17.821247 14707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "3e0da1f0c2584f36a1bca4f01e8f6ae0"
format_stamp: "Formatted at 2026-08-12 03:12:17 on dist-test-slave-zpfg"
server_key: "53861a365b3d2e3316d8021c960af9ca"
server_key_iv: "0f9f5ae654cb16d79cc79c8923e24723"
server_key_version: "encryptionkey@0"
I20260812 03:12:17.821426 14707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:17.838332 14707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:17.842095 14707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:17.842334 14707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:17.843052 14707 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:12:17.843956 14707 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:12:17.844026 14707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:17.844100 14707 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:12:17.844153 14707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:17.853626 14707 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:38701
I20260812 03:12:17.853621 14847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:38701 every 8 connection(s)
I20260812 03:12:17.854826 14707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:12:17.858777 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 14707
I20260812 03:12:17.858949 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance
I20260812 03:12:17.859228 27626 external_mini_cluster.cc:1428] Setting key 79ac301c711704193cf22836bc20d3e0
Aug 12 03:12:17 dist-test-slave-zpfg krb5kdc[14297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504337, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:12:17.869596 14385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:17.856880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:38237 (local address 127.26.250.190:38809)
0812 03:12:17.857027 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:17.857031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:17.857943 (+ 912us) server_negotiation.cc:407] Connection header received
0812 03:12:17.859553 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:17.859557 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:17.859618 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:17.859710 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:17.861122 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.861822 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.863011 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.863224 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.863313 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:17.865904 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:17.865912 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:17.865915 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:17.865941 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:17.867508 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:17.868011 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:17.868014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:17.868015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:17.868094 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:17.868474 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:17.868477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:17.868478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:17.868692 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:17.868779 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:17.869361 (+ 582us) server_negotiation.cc:299] Negotiation successful
0812 03:12:17.869467 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 03:12:17.870666 14850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:17.857292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:38809 (local address 127.26.250.131:38237)
0812 03:12:17.857777 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:17.857825 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:17.859291 (+ 1466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:17.859935 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:17.859944 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:17.860344 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:17.860978 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:17.860998 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.861939 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.861942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:17.862906 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:17.862915 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.863362 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.863365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:17.863508 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:17.864145 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:17.864161 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:17.865681 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:17.867628 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:17.867633 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:17.867646 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:17.867918 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:17.868203 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:17.868212 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:17.868214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:17.868380 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:17.868819 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:17.868825 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:17.869101 (+ 276us) client_negotiation.cc:769] Sending connection context
0812 03:12:17.869309 (+ 208us) client_negotiation.cc:240] Negotiation successful
0812 03:12:17.869612 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":123,"threads_started":1}
I20260812 03:12:17.872077 14848 heartbeater.cc:347] Connected to a master server at 127.26.250.190:38809
I20260812 03:12:17.872411 14848 heartbeater.cc:464] Registering TS with master...
I20260812 03:12:17.873063 14848 heartbeater.cc:511] Master 127.26.250.190:38809 requested a full tablet report, sending...
I20260812 03:12:17.874279 14344 ts_manager.cc:194] Registered new tserver with Master: 3e0da1f0c2584f36a1bca4f01e8f6ae0 (127.26.250.131:38701)
I20260812 03:12:17.874955 14344 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.131@KRBTEST.COM'} at 127.26.250.131:38237
I20260812 03:12:17.885772 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:12:17.896875 14385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:17.888160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44728 (local address 127.26.250.190:38809)
0812 03:12:17.888321 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:17.888324 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:17.888449 (+ 125us) server_negotiation.cc:407] Connection header received
0812 03:12:17.888559 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:17.888561 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:17.888604 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:17.888673 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:17.889377 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.889895 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.890875 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.891100 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.891303 (+ 203us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:17.892895 (+ 1592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:17.892898 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:17.892901 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:17.892928 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:17.894741 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:17.895357 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:17.895360 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:17.895361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:17.895446 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:17.895778 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:17.895781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:17.895782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:17.896030 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:17.896223 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:17.896591 (+ 368us) server_negotiation.cc:299] Negotiation successful
0812 03:12:17.896717 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 03:12:17.900529 14344 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:12:17.904018 14344 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:12:17.919632 14862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:17.913888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:33115 (local address 127.0.0.1:42032)
0812 03:12:17.914780 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:17.914794 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:17.914877 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:17.915554 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:17.915557 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:17.915573 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:17.915651 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:17.915656 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.916660 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.916666 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:17.917855 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:17.917869 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.918979 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.918983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:17.919121 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:17.919179 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 03:12:17.919309 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 03:12:17.919456 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"thread_start_us":96,"threads_started":1}
I20260812 03:12:17.919663 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:17.914332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42032 (local address 127.26.250.130:33115)
0812 03:12:17.915130 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:17.915144 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:17.915169 (+ 25us) server_negotiation.cc:407] Connection header received
0812 03:12:17.915242 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:17.915246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:17.915432 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:17.915587 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:17.915747 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.916516 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.918002 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.918836 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.918973 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:17.919116 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:17.919359 (+ 243us) server_negotiation.cc:299] Negotiation successful
0812 03:12:17.919476 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":146,"threads_started":1}
I20260812 03:12:17.920058 14859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:17.913537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:38701 (local address 127.0.0.1:51478)
0812 03:12:17.914225 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:17.914243 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:17.914378 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:17.915264 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:17.915267 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:17.915287 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:17.915423 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:17.915439 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.917116 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.917119 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:17.918488 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:17.918500 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.919627 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.919631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:17.919748 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:17.919783 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 03:12:17.919847 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 03:12:17.919921 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"mutex_wait_us":43,"thread_start_us":128,"threads_started":1}
I20260812 03:12:17.920714 14861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:17.913108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:32999 (local address 127.0.0.1:40030)
0812 03:12:17.914052 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:17.914115 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:17.914378 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:17.915213 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:17.915219 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:17.915242 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:17.915390 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:17.915398 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.917087 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.917094 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:17.918977 (+ 1883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:17.918987 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.920328 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.920334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:17.920421 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:17.920441 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 03:12:17.920505 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 03:12:17.920576 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"mutex_wait_us":5,"thread_start_us":91,"threads_started":1}
I20260812 03:12:17.920778 14860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:17.913347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40030 (local address 127.26.250.129:32999)
0812 03:12:17.913886 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:17.913894 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:17.914217 (+ 323us) server_negotiation.cc:407] Connection header received
0812 03:12:17.914601 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:17.914608 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:17.914862 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:17.914993 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:17.915532 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.916920 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.919148 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.920189 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.920341 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:17.920478 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:17.920532 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 03:12:17.920626 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":158,"threads_started":1}
I20260812 03:12:17.921321 14863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:17.914171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51478 (local address 127.26.250.131:38701)
0812 03:12:17.914755 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:17.914773 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:17.914802 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:12:17.914885 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:17.914892 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:17.915098 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:17.915274 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:17.915601 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.916920 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.918628 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:17.919477 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:17.920389 (+ 912us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:17.920538 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:17.920587 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 03:12:17.921168 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":116,"threads_started":1}
I20260812 03:12:17.923524 14629 tablet_service.cc:1467] Processing CreateTablet for tablet caa3ebadad904e9fb9fc578d57e8a70d (DEFAULT_TABLE table=test-table [id=fb120c37bdf9411eaf79ef3650cd8076]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:12:17.923749 14481 tablet_service.cc:1467] Processing CreateTablet for tablet caa3ebadad904e9fb9fc578d57e8a70d (DEFAULT_TABLE table=test-table [id=fb120c37bdf9411eaf79ef3650cd8076]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:12:17.924285 14776 tablet_service.cc:1467] Processing CreateTablet for tablet caa3ebadad904e9fb9fc578d57e8a70d (DEFAULT_TABLE table=test-table [id=fb120c37bdf9411eaf79ef3650cd8076]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:12:17.924821 14629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caa3ebadad904e9fb9fc578d57e8a70d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:17.924909 14481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caa3ebadad904e9fb9fc578d57e8a70d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:17.925299 14776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet caa3ebadad904e9fb9fc578d57e8a70d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:17.933164 14865 tablet_bootstrap.cc:492] T caa3ebadad904e9fb9fc578d57e8a70d P 538fe07ab0fd4032837ffe52d4e0f1b5: Bootstrap starting.
I20260812 03:12:17.934226 14866 tablet_bootstrap.cc:492] T caa3ebadad904e9fb9fc578d57e8a70d P d622e984b9b74ec3ac213f1d137c3b83: Bootstrap starting.
I20260812 03:12:17.934819 14867 tablet_bootstrap.cc:492] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0: Bootstrap starting.
I20260812 03:12:17.936103 14865 tablet_bootstrap.cc:654] T caa3ebadad904e9fb9fc578d57e8a70d P 538fe07ab0fd4032837ffe52d4e0f1b5: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:17.936261 14866 tablet_bootstrap.cc:654] T caa3ebadad904e9fb9fc578d57e8a70d P d622e984b9b74ec3ac213f1d137c3b83: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:17.937204 14867 tablet_bootstrap.cc:654] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:17.937225 14865 log.cc:824] T caa3ebadad904e9fb9fc578d57e8a70d P 538fe07ab0fd4032837ffe52d4e0f1b5: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:17.937358 14866 log.cc:824] T caa3ebadad904e9fb9fc578d57e8a70d P d622e984b9b74ec3ac213f1d137c3b83: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:17.938675 14867 log.cc:824] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:17.939886 14866 tablet_bootstrap.cc:492] T caa3ebadad904e9fb9fc578d57e8a70d P d622e984b9b74ec3ac213f1d137c3b83: No bootstrap required, opened a new log
I20260812 03:12:17.940065 14866 ts_tablet_manager.cc:1397] T caa3ebadad904e9fb9fc578d57e8a70d P d622e984b9b74ec3ac213f1d137c3b83: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:12:17.940116 14865 tablet_bootstrap.cc:492] T caa3ebadad904e9fb9fc578d57e8a70d P 538fe07ab0fd4032837ffe52d4e0f1b5: No bootstrap required, opened a new log
I20260812 03:12:17.940336 14865 ts_tablet_manager.cc:1397] T caa3ebadad904e9fb9fc578d57e8a70d P 538fe07ab0fd4032837ffe52d4e0f1b5: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260812 03:12:17.940917 14867 tablet_bootstrap.cc:492] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0: No bootstrap required, opened a new log
I20260812 03:12:17.941104 14867 ts_tablet_manager.cc:1397] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 03:12:17.943255 14866 raft_consensus.cc:348] T caa3ebadad904e9fb9fc578d57e8a70d P d622e984b9b74ec3ac213f1d137c3b83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538fe07ab0fd4032837ffe52d4e0f1b5" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 32999 } } peers { permanent_uuid: "3e0da1f0c2584f36a1bca4f01e8f6ae0" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38701 } } peers { permanent_uuid: "d622e984b9b74ec3ac213f1d137c3b83" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 33115 } }
I20260812 03:12:17.943548 14866 raft_consensus.cc:374] T caa3ebadad904e9fb9fc578d57e8a70d P d622e984b9b74ec3ac213f1d137c3b83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:17.943606 14866 raft_consensus.cc:729] T caa3ebadad904e9fb9fc578d57e8a70d P d622e984b9b74ec3ac213f1d137c3b83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d622e984b9b74ec3ac213f1d137c3b83, State: Initialized, Role: FOLLOWER
I20260812 03:12:17.944101 14866 consensus_queue.cc:260] T caa3ebadad904e9fb9fc578d57e8a70d P d622e984b9b74ec3ac213f1d137c3b83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538fe07ab0fd4032837ffe52d4e0f1b5" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 32999 } } peers { permanent_uuid: "3e0da1f0c2584f36a1bca4f01e8f6ae0" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38701 } } peers { permanent_uuid: "d622e984b9b74ec3ac213f1d137c3b83" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 33115 } }
I20260812 03:12:17.944886 14702 heartbeater.cc:503] Master 127.26.250.190:38809 was elected leader, sending a full tablet report...
I20260812 03:12:17.945266 14866 ts_tablet_manager.cc:1428] T caa3ebadad904e9fb9fc578d57e8a70d P d622e984b9b74ec3ac213f1d137c3b83: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260812 03:12:17.945681 14867 raft_consensus.cc:348] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538fe07ab0fd4032837ffe52d4e0f1b5" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 32999 } } peers { permanent_uuid: "3e0da1f0c2584f36a1bca4f01e8f6ae0" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38701 } } peers { permanent_uuid: "d622e984b9b74ec3ac213f1d137c3b83" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 33115 } }
I20260812 03:12:17.945678 14865 raft_consensus.cc:348] T caa3ebadad904e9fb9fc578d57e8a70d P 538fe07ab0fd4032837ffe52d4e0f1b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538fe07ab0fd4032837ffe52d4e0f1b5" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 32999 } } peers { permanent_uuid: "3e0da1f0c2584f36a1bca4f01e8f6ae0" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38701 } } peers { permanent_uuid: "d622e984b9b74ec3ac213f1d137c3b83" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 33115 } }
I20260812 03:12:17.945987 14867 raft_consensus.cc:374] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:17.945987 14865 raft_consensus.cc:374] T caa3ebadad904e9fb9fc578d57e8a70d P 538fe07ab0fd4032837ffe52d4e0f1b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:17.946046 14867 raft_consensus.cc:729] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e0da1f0c2584f36a1bca4f01e8f6ae0, State: Initialized, Role: FOLLOWER
I20260812 03:12:17.946046 14865 raft_consensus.cc:729] T caa3ebadad904e9fb9fc578d57e8a70d P 538fe07ab0fd4032837ffe52d4e0f1b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 538fe07ab0fd4032837ffe52d4e0f1b5, State: Initialized, Role: FOLLOWER
I20260812 03:12:17.946619 14867 consensus_queue.cc:260] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538fe07ab0fd4032837ffe52d4e0f1b5" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 32999 } } peers { permanent_uuid: "3e0da1f0c2584f36a1bca4f01e8f6ae0" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38701 } } peers { permanent_uuid: "d622e984b9b74ec3ac213f1d137c3b83" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 33115 } }
I20260812 03:12:17.947538 14848 heartbeater.cc:503] Master 127.26.250.190:38809 was elected leader, sending a full tablet report...
I20260812 03:12:17.947891 14867 ts_tablet_manager.cc:1428] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260812 03:12:17.948757 14865 consensus_queue.cc:260] T caa3ebadad904e9fb9fc578d57e8a70d P 538fe07ab0fd4032837ffe52d4e0f1b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538fe07ab0fd4032837ffe52d4e0f1b5" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 32999 } } peers { permanent_uuid: "3e0da1f0c2584f36a1bca4f01e8f6ae0" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38701 } } peers { permanent_uuid: "d622e984b9b74ec3ac213f1d137c3b83" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 33115 } }
I20260812 03:12:17.949551 14556 heartbeater.cc:503] Master 127.26.250.190:38809 was elected leader, sending a full tablet report...
I20260812 03:12:17.951061 14865 ts_tablet_manager.cc:1428] T caa3ebadad904e9fb9fc578d57e8a70d P 538fe07ab0fd4032837ffe52d4e0f1b5: Time spent starting tablet: real 0.011s user 0.007s sys 0.002s
W20260812 03:12:18.105944 14849 tablet.cc:2369] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:12:18.111972 14557 tablet.cc:2369] T caa3ebadad904e9fb9fc578d57e8a70d P 538fe07ab0fd4032837ffe52d4e0f1b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 03:12:18.113967 14703 tablet.cc:2369] T caa3ebadad904e9fb9fc578d57e8a70d P d622e984b9b74ec3ac213f1d137c3b83: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:12:18.121061 14872 raft_consensus.cc:482] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:12:18.121264 14872 raft_consensus.cc:504] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538fe07ab0fd4032837ffe52d4e0f1b5" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 32999 } } peers { permanent_uuid: "3e0da1f0c2584f36a1bca4f01e8f6ae0" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38701 } } peers { permanent_uuid: "d622e984b9b74ec3ac213f1d137c3b83" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 33115 } }
I20260812 03:12:18.122608 14872 leader_election.cc:288] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 538fe07ab0fd4032837ffe52d4e0f1b5 (127.26.250.129:32999), d622e984b9b74ec3ac213f1d137c3b83 (127.26.250.130:33115)
I20260812 03:12:18.126987 14860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:18.123141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:47361 (local address 127.26.250.129:32999)
0812 03:12:18.123281 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:18.123304 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:18.123328 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:12:18.123393 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:18.123396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:18.123455 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:18.123562 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:18.123886 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:18.124402 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:18.125694 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:18.126412 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:18.126530 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:18.126587 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:18.126777 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 03:12:18.126866 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:12:18.126986 14850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:18.123025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:32999 (local address 127.26.250.131:47361)
0812 03:12:18.123166 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:18.123183 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:18.123297 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:18.123579 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:18.123582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:18.123603 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:18.123747 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:18.123753 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:18.124522 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:18.124527 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:18.125561 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:18.125571 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:18.126522 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:18.126526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:18.126636 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:18.126663 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 03:12:18.126758 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 03:12:18.126838 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260812 03:12:18.127658 14874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:18.123351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:33115 (local address 127.26.250.131:35909)
0812 03:12:18.123809 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:18.123820 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:18.123939 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:18.124304 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:18.124307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:18.124326 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:18.124476 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:18.124483 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:18.125312 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:18.125317 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:18.126321 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:18.126331 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:18.127262 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:18.127268 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:18.127370 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:18.127394 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:12:18.127474 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 03:12:18.127548 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":168,"threads_started":1}
I20260812 03:12:18.127712 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:18.123517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:35909 (local address 127.26.250.130:33115)
0812 03:12:18.123701 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:18.123706 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:18.123902 (+ 196us) server_negotiation.cc:407] Connection header received
0812 03:12:18.124038 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:18.124041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:18.124087 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:18.124287 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:18.124601 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:18.125201 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:18.126485 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:18.127117 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:18.127252 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:18.127299 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:18.127502 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 03:12:18.127586 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 03:12:18.127818 14504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caa3ebadad904e9fb9fc578d57e8a70d" candidate_uuid: "3e0da1f0c2584f36a1bca4f01e8f6ae0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "538fe07ab0fd4032837ffe52d4e0f1b5" is_pre_election: true
I20260812 03:12:18.128144 14504 raft_consensus.cc:2454] T caa3ebadad904e9fb9fc578d57e8a70d P 538fe07ab0fd4032837ffe52d4e0f1b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e0da1f0c2584f36a1bca4f01e8f6ae0 in term 0.
I20260812 03:12:18.128602 14651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caa3ebadad904e9fb9fc578d57e8a70d" candidate_uuid: "3e0da1f0c2584f36a1bca4f01e8f6ae0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d622e984b9b74ec3ac213f1d137c3b83" is_pre_election: true
I20260812 03:12:18.128717 14725 leader_election.cc:302] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e0da1f0c2584f36a1bca4f01e8f6ae0, 538fe07ab0fd4032837ffe52d4e0f1b5; no voters:
I20260812 03:12:18.128922 14651 raft_consensus.cc:2454] T caa3ebadad904e9fb9fc578d57e8a70d P d622e984b9b74ec3ac213f1d137c3b83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e0da1f0c2584f36a1bca4f01e8f6ae0 in term 0.
I20260812 03:12:18.129068 14872 raft_consensus.cc:2781] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:12:18.129185 14872 raft_consensus.cc:482] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:12:18.129235 14872 raft_consensus.cc:3037] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:18.130424 14872 raft_consensus.cc:504] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538fe07ab0fd4032837ffe52d4e0f1b5" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 32999 } } peers { permanent_uuid: "3e0da1f0c2584f36a1bca4f01e8f6ae0" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38701 } } peers { permanent_uuid: "d622e984b9b74ec3ac213f1d137c3b83" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 33115 } }
I20260812 03:12:18.130854 14872 leader_election.cc:288] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0 [CANDIDATE]: Term 1 election: Requested vote from peers 538fe07ab0fd4032837ffe52d4e0f1b5 (127.26.250.129:32999), d622e984b9b74ec3ac213f1d137c3b83 (127.26.250.130:33115)
I20260812 03:12:18.131239 14504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caa3ebadad904e9fb9fc578d57e8a70d" candidate_uuid: "3e0da1f0c2584f36a1bca4f01e8f6ae0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "538fe07ab0fd4032837ffe52d4e0f1b5"
I20260812 03:12:18.131347 14651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "caa3ebadad904e9fb9fc578d57e8a70d" candidate_uuid: "3e0da1f0c2584f36a1bca4f01e8f6ae0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d622e984b9b74ec3ac213f1d137c3b83"
I20260812 03:12:18.131405 14504 raft_consensus.cc:3037] T caa3ebadad904e9fb9fc578d57e8a70d P 538fe07ab0fd4032837ffe52d4e0f1b5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:18.131469 14651 raft_consensus.cc:3037] T caa3ebadad904e9fb9fc578d57e8a70d P d622e984b9b74ec3ac213f1d137c3b83 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:18.132877 14651 raft_consensus.cc:2454] T caa3ebadad904e9fb9fc578d57e8a70d P d622e984b9b74ec3ac213f1d137c3b83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e0da1f0c2584f36a1bca4f01e8f6ae0 in term 1.
I20260812 03:12:18.132877 14504 raft_consensus.cc:2454] T caa3ebadad904e9fb9fc578d57e8a70d P 538fe07ab0fd4032837ffe52d4e0f1b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e0da1f0c2584f36a1bca4f01e8f6ae0 in term 1.
I20260812 03:12:18.133256 14725 leader_election.cc:302] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e0da1f0c2584f36a1bca4f01e8f6ae0, 538fe07ab0fd4032837ffe52d4e0f1b5; no voters:
I20260812 03:12:18.133464 14872 raft_consensus.cc:2781] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:18.133718 14872 raft_consensus.cc:686] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0 [term 1 LEADER]: Becoming Leader. State: Replica: 3e0da1f0c2584f36a1bca4f01e8f6ae0, State: Running, Role: LEADER
I20260812 03:12:18.134081 14872 consensus_queue.cc:237] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538fe07ab0fd4032837ffe52d4e0f1b5" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 32999 } } peers { permanent_uuid: "3e0da1f0c2584f36a1bca4f01e8f6ae0" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38701 } } peers { permanent_uuid: "d622e984b9b74ec3ac213f1d137c3b83" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 33115 } }
I20260812 03:12:18.137742 14343 catalog_manager.cc:5495] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e0da1f0c2584f36a1bca4f01e8f6ae0 (127.26.250.131). New cstate: current_term: 1 leader_uuid: "3e0da1f0c2584f36a1bca4f01e8f6ae0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "538fe07ab0fd4032837ffe52d4e0f1b5" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 32999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e0da1f0c2584f36a1bca4f01e8f6ae0" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d622e984b9b74ec3ac213f1d137c3b83" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 33115 } health_report { overall_health: UNKNOWN } } }
I20260812 03:12:18.165114 14863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:18.160674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51482 (local address 127.26.250.131:38701)
0812 03:12:18.160838 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:18.160843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:18.160861 (+ 18us) server_negotiation.cc:407] Connection header received
0812 03:12:18.161013 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:18.161015 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:18.161079 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:18.161297 (+ 218us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:12:18.161596 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:18.162168 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:18.163233 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:18.163487 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:18.163621 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:18.164045 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:12:18.164527 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:12:18.164727 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:18.164869 (+ 142us) server_negotiation.cc:299] Negotiation successful
0812 03:12:18.164918 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 03:12:18.172360 14651 raft_consensus.cc:1261] T caa3ebadad904e9fb9fc578d57e8a70d P d622e984b9b74ec3ac213f1d137c3b83 [term 1 FOLLOWER]: Refusing update from remote peer 3e0da1f0c2584f36a1bca4f01e8f6ae0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:12:18.172370 14504 raft_consensus.cc:1261] T caa3ebadad904e9fb9fc578d57e8a70d P 538fe07ab0fd4032837ffe52d4e0f1b5 [term 1 FOLLOWER]: Refusing update from remote peer 3e0da1f0c2584f36a1bca4f01e8f6ae0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:12:18.173235 14875 consensus_queue.cc:1035] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d622e984b9b74ec3ac213f1d137c3b83" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 33115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:12:18.173444 14872 consensus_queue.cc:1035] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "538fe07ab0fd4032837ffe52d4e0f1b5" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 32999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:12:18.182886 14880 mvcc.cc:205] Tried to move back new op lower bound from 7317521769144705024 to 7317521769000144896. Current Snapshot: MvccSnapshot[applied={T|T < 7317521769144705024}]
I20260812 03:12:18.184118 14879 mvcc.cc:205] Tried to move back new op lower bound from 7317521769144705024 to 7317521769000144896. Current Snapshot: MvccSnapshot[applied={T|T < 7317521769144705024}]
I20260812 03:12:18.191434 14343 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:12:18.191615 14343 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:12:18.194393 14343 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 75de098b2947496bac7042b71ea2e8b0: Sending DeleteTablet for 3 replicas of tablet caa3ebadad904e9fb9fc578d57e8a70d
I20260812 03:12:18.195363 14776 tablet_service.cc:1514] Processing DeleteTablet for tablet caa3ebadad904e9fb9fc578d57e8a70d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:12:18 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:51478
I20260812 03:12:18.195684 14481 tablet_service.cc:1514] Processing DeleteTablet for tablet caa3ebadad904e9fb9fc578d57e8a70d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:12:18 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:40030
I20260812 03:12:18.196043 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 14411
I20260812 03:12:18.196202 14629 tablet_service.cc:1514] Processing DeleteTablet for tablet caa3ebadad904e9fb9fc578d57e8a70d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:12:18 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:42032
I20260812 03:12:18.197080 14888 tablet_replica.cc:330] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0: stopping tablet replica
I20260812 03:12:18.197490 14888 raft_consensus.cc:2229] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:12:18.198005 14888 raft_consensus.cc:2258] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:12:18.201357 14891 tablet_replica.cc:330] T caa3ebadad904e9fb9fc578d57e8a70d P d622e984b9b74ec3ac213f1d137c3b83: stopping tablet replica
I20260812 03:12:18.201821 14891 raft_consensus.cc:2229] T caa3ebadad904e9fb9fc578d57e8a70d P d622e984b9b74ec3ac213f1d137c3b83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:12:18.202284 14891 raft_consensus.cc:2258] T caa3ebadad904e9fb9fc578d57e8a70d P d622e984b9b74ec3ac213f1d137c3b83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:12:18.203418 14888 ts_tablet_manager.cc:1905] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:12:18.204691 14891 ts_tablet_manager.cc:1905] T caa3ebadad904e9fb9fc578d57e8a70d P d622e984b9b74ec3ac213f1d137c3b83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:12:18.206655 14888 ts_tablet_manager.cc:1918] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:12:18.206764 14888 log.cc:1196] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/caa3ebadad904e9fb9fc578d57e8a70d
I20260812 03:12:18.207144 14888 ts_tablet_manager.cc:1939] T caa3ebadad904e9fb9fc578d57e8a70d P 3e0da1f0c2584f36a1bca4f01e8f6ae0: Deleting consensus metadata
I20260812 03:12:18.207930 14891 ts_tablet_manager.cc:1918] T caa3ebadad904e9fb9fc578d57e8a70d P d622e984b9b74ec3ac213f1d137c3b83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:12:18.208029 14891 log.cc:1196] T caa3ebadad904e9fb9fc578d57e8a70d P d622e984b9b74ec3ac213f1d137c3b83: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786504075277362-27626-0/minicluster-data/ts-1/wal/wals/caa3ebadad904e9fb9fc578d57e8a70d
I20260812 03:12:18.208379 14891 ts_tablet_manager.cc:1939] T caa3ebadad904e9fb9fc578d57e8a70d P d622e984b9b74ec3ac213f1d137c3b83: Deleting consensus metadata
I20260812 03:12:18.209060 14329 catalog_manager.cc:4841] TS 3e0da1f0c2584f36a1bca4f01e8f6ae0 (127.26.250.131:38701): tablet caa3ebadad904e9fb9fc578d57e8a70d (table test-table [id=fb120c37bdf9411eaf79ef3650cd8076]) successfully deleted
I20260812 03:12:18.209581 14327 catalog_manager.cc:4841] TS d622e984b9b74ec3ac213f1d137c3b83 (127.26.250.130:33115): tablet caa3ebadad904e9fb9fc578d57e8a70d (table test-table [id=fb120c37bdf9411eaf79ef3650cd8076]) successfully deleted
W20260812 03:12:18.214680 14327 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.250.129:32999 (error 108)
I20260812 03:12:18.214942 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 14561
I20260812 03:12:18.218587 14861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:18.218125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:32999 (local address 127.0.0.1:40040)
0812 03:12:18.218327 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:18.218463 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.129:32999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260812 03:12:18.218899 14327 catalog_manager.cc:4584] TS 538fe07ab0fd4032837ffe52d4e0f1b5 (127.26.250.129:32999): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet caa3ebadad904e9fb9fc578d57e8a70d: Network error: Client connection negotiation failed: client connection to 127.26.250.129:32999: connect: Connection refused (error 111)
I20260812 03:12:18.224081 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 14707
I20260812 03:12:18.233299 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 14310
2026-08-12T03:12:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2637 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T03:12:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:12:18Z Disabled control of system clock
I20260812 03:12:18.271284 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:39137
--webserver_interface=127.26.250.190
--webserver_port=0
--builtin_ntp_servers=127.26.250.148:43129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:39137
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:12:18.392494 14896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:18.392796 14896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:18.392838 14896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:18.397037 14896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:18.397114 14896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:18.397130 14896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:18.397148 14896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:18.397164 14896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:18.402139 14896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:43129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:39137
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:39137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:18.403312 14896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:18.404706 14896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:18.413987 14901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:18.414068 14896 server_base.cc:1034] running on GCE node
W20260812 03:12:18.414005 14902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:18.413975 14904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:18.415032 14896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:18.416249 14896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:18.417428 14896 hybrid_clock.cc:648] HybridClock initialized: now 1786504338417401 us; error 48 us; skew 500 ppm
I20260812 03:12:18.419620 14896 webserver.cc:460] Webserver started at http://127.26.250.190:41109/ using document root <none> and password file <none>
I20260812 03:12:18.420245 14896 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:18.420311 14896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:18.420543 14896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:18.422914 14896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "972eb1d4ac37490797798f4cb13e8bf3"
format_stamp: "Formatted at 2026-08-12 03:12:18 on dist-test-slave-zpfg"
server_key: "00e68673919115bb2ba3100efeabae76"
server_key_iv: "3d214e60fccb252e37f19d175858c77a"
server_key_version: "encryptionkey@0"
I20260812 03:12:18.423516 14896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "972eb1d4ac37490797798f4cb13e8bf3"
format_stamp: "Formatted at 2026-08-12 03:12:18 on dist-test-slave-zpfg"
server_key: "00e68673919115bb2ba3100efeabae76"
server_key_iv: "3d214e60fccb252e37f19d175858c77a"
server_key_version: "encryptionkey@0"
I20260812 03:12:18.427623 14896 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260812 03:12:18.430567 14910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:18.432090 14896 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 03:12:18.432277 14896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "972eb1d4ac37490797798f4cb13e8bf3"
format_stamp: "Formatted at 2026-08-12 03:12:18 on dist-test-slave-zpfg"
server_key: "00e68673919115bb2ba3100efeabae76"
server_key_iv: "3d214e60fccb252e37f19d175858c77a"
server_key_version: "encryptionkey@0"
I20260812 03:12:18.432425 14896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:18.461706 14896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:18.462596 14896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:18.462831 14896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:18.470520 14896 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:39137
I20260812 03:12:18.470535 14965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:39137 every 8 connection(s)
I20260812 03:12:18.471658 14896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:12:18.475059 14966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:18.479435 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 14896
I20260812 03:12:18.479553 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:12:18.479902 27626 external_mini_cluster.cc:1428] Setting key 2accac59bbbb3f9101893a24d481845c
I20260812 03:12:18.482004 14966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 972eb1d4ac37490797798f4cb13e8bf3: Bootstrap starting.
I20260812 03:12:18.484457 14966 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 972eb1d4ac37490797798f4cb13e8bf3: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:18.485265 14966 log.cc:824] T 00000000000000000000000000000000 P 972eb1d4ac37490797798f4cb13e8bf3: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:18.487408 14966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 972eb1d4ac37490797798f4cb13e8bf3: No bootstrap required, opened a new log
I20260812 03:12:18.490233 14969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:18.481512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34054 (local address 127.26.250.190:39137)
0812 03:12:18.482294 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:18.482319 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:18.482396 (+ 77us) server_negotiation.cc:407] Connection header received
0812 03:12:18.483126 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:18.483145 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:18.483578 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:18.484000 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:18.484481 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:18.485609 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:18.486640 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:18.487154 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:18.487334 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:18.488297 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:18.488302 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:18.488324 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:18.488772 (+ 448us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:18.488831 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:18.488882 (+ 51us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:18.489002 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:18.489163 (+ 161us) server_negotiation.cc:299] Negotiation successful
0812 03:12:18.489434 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":154,"threads_started":1}
I20260812 03:12:18.491496 14966 raft_consensus.cc:348] T 00000000000000000000000000000000 P 972eb1d4ac37490797798f4cb13e8bf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "972eb1d4ac37490797798f4cb13e8bf3" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39137 } }
I20260812 03:12:18.491863 14966 raft_consensus.cc:374] T 00000000000000000000000000000000 P 972eb1d4ac37490797798f4cb13e8bf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:18.491962 14966 raft_consensus.cc:729] T 00000000000000000000000000000000 P 972eb1d4ac37490797798f4cb13e8bf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972eb1d4ac37490797798f4cb13e8bf3, State: Initialized, Role: FOLLOWER
I20260812 03:12:18.492470 14966 consensus_queue.cc:260] T 00000000000000000000000000000000 P 972eb1d4ac37490797798f4cb13e8bf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "972eb1d4ac37490797798f4cb13e8bf3" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39137 } }
I20260812 03:12:18.492663 14966 raft_consensus.cc:388] T 00000000000000000000000000000000 P 972eb1d4ac37490797798f4cb13e8bf3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:12:18.492743 14966 raft_consensus.cc:482] T 00000000000000000000000000000000 P 972eb1d4ac37490797798f4cb13e8bf3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:12:18.492856 14966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 972eb1d4ac37490797798f4cb13e8bf3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:18.493893 14966 raft_consensus.cc:504] T 00000000000000000000000000000000 P 972eb1d4ac37490797798f4cb13e8bf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "972eb1d4ac37490797798f4cb13e8bf3" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39137 } }
I20260812 03:12:18.494280 14966 leader_election.cc:302] T 00000000000000000000000000000000 P 972eb1d4ac37490797798f4cb13e8bf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 972eb1d4ac37490797798f4cb13e8bf3; no voters:
I20260812 03:12:18.494688 14966 leader_election.cc:288] T 00000000000000000000000000000000 P 972eb1d4ac37490797798f4cb13e8bf3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:12:18.494830 14971 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 972eb1d4ac37490797798f4cb13e8bf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:18.495111 14971 raft_consensus.cc:686] T 00000000000000000000000000000000 P 972eb1d4ac37490797798f4cb13e8bf3 [term 1 LEADER]: Becoming Leader. State: Replica: 972eb1d4ac37490797798f4cb13e8bf3, State: Running, Role: LEADER
I20260812 03:12:18.495457 14971 consensus_queue.cc:237] T 00000000000000000000000000000000 P 972eb1d4ac37490797798f4cb13e8bf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "972eb1d4ac37490797798f4cb13e8bf3" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39137 } }
I20260812 03:12:18.496002 14966 sys_catalog.cc:564] T 00000000000000000000000000000000 P 972eb1d4ac37490797798f4cb13e8bf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:18.497146 14972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 972eb1d4ac37490797798f4cb13e8bf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "972eb1d4ac37490797798f4cb13e8bf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "972eb1d4ac37490797798f4cb13e8bf3" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39137 } } }
I20260812 03:12:18.497219 14973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 972eb1d4ac37490797798f4cb13e8bf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 972eb1d4ac37490797798f4cb13e8bf3. Latest consensus state: current_term: 1 leader_uuid: "972eb1d4ac37490797798f4cb13e8bf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "972eb1d4ac37490797798f4cb13e8bf3" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 39137 } } }
I20260812 03:12:18.497295 14972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 972eb1d4ac37490797798f4cb13e8bf3 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:18.497339 14973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 972eb1d4ac37490797798f4cb13e8bf3 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:18.497684 14980 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:12:18.500985 14980 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:12:18.507121 14980 catalog_manager.cc:1347] Generated new cluster ID: 1269fd8b33184d1b9d55d50847b49e3b
I20260812 03:12:18.507203 14980 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:12:18.539508 14980 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:12:18.540748 14980 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:12:18.558997 14980 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 972eb1d4ac37490797798f4cb13e8bf3: Generated new TSK 0
I20260812 03:12:18.559928 14980 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:12:18.566936 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:0
--local_ip_for_outbound_sockets=127.26.250.129
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39137
--builtin_ntp_servers=127.26.250.148:43129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:12:18.688969 14992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:18.689299 14992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:18.689390 14992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:18.693660 14992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:18.693755 14992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:18.693868 14992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:12:18.699321 14992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:43129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:18.700575 14992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:18.702111 14992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:18.712864 15000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:18.712989 14998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:18.713050 14997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:18.713140 14992 server_base.cc:1034] running on GCE node
I20260812 03:12:18.713742 14992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:18.714556 14992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:18.715770 14992 hybrid_clock.cc:648] HybridClock initialized: now 1786504338715741 us; error 49 us; skew 500 ppm
I20260812 03:12:18.717908 14992 webserver.cc:460] Webserver started at http://127.26.250.129:34945/ using document root <none> and password file <none>
I20260812 03:12:18.718636 14992 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:18.718752 14992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:18.719014 14992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:18.721390 14992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-0/data/instance:
uuid: "bc4c52bc738c4c67bd306febb6186c44"
format_stamp: "Formatted at 2026-08-12 03:12:18 on dist-test-slave-zpfg"
server_key: "6a5e3914bc9da4f627bf978bbc4270dd"
server_key_iv: "91a862962e8502309491fe200e159826"
server_key_version: "encryptionkey@0"
I20260812 03:12:18.721998 14992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance:
uuid: "bc4c52bc738c4c67bd306febb6186c44"
format_stamp: "Formatted at 2026-08-12 03:12:18 on dist-test-slave-zpfg"
server_key: "6a5e3914bc9da4f627bf978bbc4270dd"
server_key_iv: "91a862962e8502309491fe200e159826"
server_key_version: "encryptionkey@0"
I20260812 03:12:18.726586 14992 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:12:18.729327 15006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:18.730901 14992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:12:18.731076 14992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "bc4c52bc738c4c67bd306febb6186c44"
format_stamp: "Formatted at 2026-08-12 03:12:18 on dist-test-slave-zpfg"
server_key: "6a5e3914bc9da4f627bf978bbc4270dd"
server_key_iv: "91a862962e8502309491fe200e159826"
server_key_version: "encryptionkey@0"
I20260812 03:12:18.731221 14992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:18.753993 14992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:18.754940 14992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:18.755198 14992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:18.755826 14992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:12:18.756884 14992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:12:18.756935 14992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:18.756976 14992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:12:18.756992 14992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:18.766578 14992 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:42495
I20260812 03:12:18.766623 15127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:42495 every 8 connection(s)
I20260812 03:12:18.767627 14992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:12:18.775682 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 14992
I20260812 03:12:18.775878 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance
I20260812 03:12:18.776292 27626 external_mini_cluster.cc:1428] Setting key 4074133e96b78edc0d95bda196685af7
I20260812 03:12:18.778904 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:0
--local_ip_for_outbound_sockets=127.26.250.130
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39137
--builtin_ntp_servers=127.26.250.148:43129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 03:12:18.780128 14969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:18.769776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:36283 (local address 127.26.250.190:39137)
0812 03:12:18.769940 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:18.769944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:18.771115 (+ 1171us) server_negotiation.cc:407] Connection header received
0812 03:12:18.772614 (+ 1499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:18.772617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:18.772682 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:18.772800 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:18.774111 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:18.774937 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:18.777056 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:18.777527 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:18.777659 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:18.779431 (+ 1772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:18.779437 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:18.779443 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:18.779462 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:18.779490 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:18.779504 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:18.779556 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:18.779805 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 03:12:18.779894 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 03:12:18.782064 15130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:18.770162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39137 (local address 127.26.250.129:36283)
0812 03:12:18.770939 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:18.770983 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:18.772367 (+ 1384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:18.772999 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:18.773007 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:18.773098 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:18.773887 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:18.773914 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:18.775091 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:18.775097 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:18.776870 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:18.776888 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:18.777755 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:18.777763 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:18.778087 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:18.778881 (+ 794us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:12:18.778883 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:18.778951 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:12:18.779221 (+ 270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:18.779627 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:18.779640 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:18.779650 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 03:12:18.780423 (+ 773us) client_negotiation.cc:240] Negotiation successful
0812 03:12:18.780811 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":189,"threads_started":1}
I20260812 03:12:18.783718 15128 heartbeater.cc:347] Connected to a master server at 127.26.250.190:39137
I20260812 03:12:18.784008 15128 heartbeater.cc:464] Registering TS with master...
I20260812 03:12:18.784614 15128 heartbeater.cc:511] Master 127.26.250.190:39137 requested a full tablet report, sending...
I20260812 03:12:18.786441 14930 ts_manager.cc:194] Registered new tserver with Master: bc4c52bc738c4c67bd306febb6186c44 (127.26.250.129:42495)
I20260812 03:12:18.787645 14930 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.250.129:36283
W20260812 03:12:18.902248 15131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:18.902606 15131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:18.902680 15131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:18.906941 15131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:18.907043 15131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:18.907142 15131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:12:18.912127 15131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:43129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:18.913357 15131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:18.914880 15131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:18.925977 15139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:18.926038 15131 server_base.cc:1034] running on GCE node
W20260812 03:12:18.925983 15136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:18.925983 15137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:18.926736 15131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:18.927480 15131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:18.928678 15131 hybrid_clock.cc:648] HybridClock initialized: now 1786504338928628 us; error 63 us; skew 500 ppm
I20260812 03:12:18.931056 15131 webserver.cc:460] Webserver started at http://127.26.250.130:37001/ using document root <none> and password file <none>
I20260812 03:12:18.931733 15131 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:18.931806 15131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:18.932045 15131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:18.934376 15131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-1/data/instance:
uuid: "74bb6887393640dbab379f9af772967f"
format_stamp: "Formatted at 2026-08-12 03:12:18 on dist-test-slave-zpfg"
server_key: "ed1adca76f949c70ff9287cd49ba75ad"
server_key_iv: "c6f4fa7228bc861b39c79393cca6b8b0"
server_key_version: "encryptionkey@0"
I20260812 03:12:18.935034 15131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance:
uuid: "74bb6887393640dbab379f9af772967f"
format_stamp: "Formatted at 2026-08-12 03:12:18 on dist-test-slave-zpfg"
server_key: "ed1adca76f949c70ff9287cd49ba75ad"
server_key_iv: "c6f4fa7228bc861b39c79393cca6b8b0"
server_key_version: "encryptionkey@0"
I20260812 03:12:18.939170 15131 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:12:18.941814 15147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:18.943491 15131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:12:18.943637 15131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "74bb6887393640dbab379f9af772967f"
format_stamp: "Formatted at 2026-08-12 03:12:18 on dist-test-slave-zpfg"
server_key: "ed1adca76f949c70ff9287cd49ba75ad"
server_key_iv: "c6f4fa7228bc861b39c79393cca6b8b0"
server_key_version: "encryptionkey@0"
I20260812 03:12:18.943776 15131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:18.963935 15131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:18.964737 15131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:18.964953 15131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:18.965643 15131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:12:18.966737 15131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:12:18.966789 15131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:18.966862 15131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:12:18.966897 15131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:18.976473 15131 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:34063
I20260812 03:12:18.976495 15268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:34063 every 8 connection(s)
I20260812 03:12:18.977463 15131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:12:18.978276 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 15131
I20260812 03:12:18.978446 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance
I20260812 03:12:18.978823 27626 external_mini_cluster.cc:1428] Setting key c730f68d45beb65ad5b8ade763905f87
I20260812 03:12:18.981102 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:0
--local_ip_for_outbound_sockets=127.26.250.131
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39137
--builtin_ntp_servers=127.26.250.148:43129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 03:12:18.988322 14969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:18.979666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:45285 (local address 127.26.250.190:39137)
0812 03:12:18.979809 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:18.979813 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:18.981185 (+ 1372us) server_negotiation.cc:407] Connection header received
0812 03:12:18.982840 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:18.982847 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:18.982912 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:18.982989 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:18.984331 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:18.984873 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:18.986074 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:18.986440 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:18.986550 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:18.987769 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:18.987772 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:18.987775 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:18.987796 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:18.987817 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:18.987825 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:18.987932 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:18.988105 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 03:12:18.988171 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 03:12:18.989331 15271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:18.980188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39137 (local address 127.26.250.130:45285)
0812 03:12:18.980999 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:18.981054 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:18.982558 (+ 1504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:18.983240 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:18.983252 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:18.983374 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:18.984163 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:18.984187 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:18.985016 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:18.985019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:18.985939 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:18.985949 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:18.986612 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:18.986617 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:18.986782 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:18.987487 (+ 705us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:12:18.987489 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:18.987531 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:12:18.987599 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:18.987939 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:18.987952 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:18.987960 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 03:12:18.988095 (+ 135us) client_negotiation.cc:240] Negotiation successful
0812 03:12:18.988414 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":156,"threads_started":1}
I20260812 03:12:18.990836 15269 heartbeater.cc:347] Connected to a master server at 127.26.250.190:39137
I20260812 03:12:18.991101 15269 heartbeater.cc:464] Registering TS with master...
I20260812 03:12:18.991659 15269 heartbeater.cc:511] Master 127.26.250.190:39137 requested a full tablet report, sending...
I20260812 03:12:18.992826 14930 ts_manager.cc:194] Registered new tserver with Master: 74bb6887393640dbab379f9af772967f (127.26.250.130:34063)
I20260812 03:12:18.993701 14930 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.250.130:45285
W20260812 03:12:19.105379 15272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:19.105664 15272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:19.105703 15272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:19.109874 15272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:19.109936 15272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:19.109995 15272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:12:19.114907 15272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:43129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:39137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:19.116047 15272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:19.117482 15272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:19.127697 15282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:19.127777 15279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:19.127765 15277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:19.128022 15272 server_base.cc:1034] running on GCE node
I20260812 03:12:19.128480 15272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:19.129146 15272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:19.130383 15272 hybrid_clock.cc:648] HybridClock initialized: now 1786504339130326 us; error 36 us; skew 500 ppm
I20260812 03:12:19.132381 15272 webserver.cc:460] Webserver started at http://127.26.250.131:34025/ using document root <none> and password file <none>
I20260812 03:12:19.133010 15272 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:19.133111 15272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:19.133396 15272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:19.135833 15272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-2/data/instance:
uuid: "94b72db7370c4b63b255e816f5f8fe57"
format_stamp: "Formatted at 2026-08-12 03:12:19 on dist-test-slave-zpfg"
server_key: "a3ec07f110e06cd9747f4c47a524aea3"
server_key_iv: "dc44f2258afff2dbcb1395e2ebfc8634"
server_key_version: "encryptionkey@0"
I20260812 03:12:19.136583 15272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance:
uuid: "94b72db7370c4b63b255e816f5f8fe57"
format_stamp: "Formatted at 2026-08-12 03:12:19 on dist-test-slave-zpfg"
server_key: "a3ec07f110e06cd9747f4c47a524aea3"
server_key_iv: "dc44f2258afff2dbcb1395e2ebfc8634"
server_key_version: "encryptionkey@0"
I20260812 03:12:19.140816 15272 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:12:19.143868 15289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:19.145285 15272 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:12:19.145478 15272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "94b72db7370c4b63b255e816f5f8fe57"
format_stamp: "Formatted at 2026-08-12 03:12:19 on dist-test-slave-zpfg"
server_key: "a3ec07f110e06cd9747f4c47a524aea3"
server_key_iv: "dc44f2258afff2dbcb1395e2ebfc8634"
server_key_version: "encryptionkey@0"
I20260812 03:12:19.145609 15272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:19.161670 15272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:19.162719 15272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:19.162966 15272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:19.163704 15272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:12:19.164810 15272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:12:19.164893 15272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:19.164978 15272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:12:19.165028 15272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:19.175629 15272 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:43971
I20260812 03:12:19.175681 15409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:43971 every 8 connection(s)
I20260812 03:12:19.176769 15272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:12:19.179430 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 15272
I20260812 03:12:19.179559 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance
I20260812 03:12:19.179942 27626 external_mini_cluster.cc:1428] Setting key 89c62ddb3aca46f35e55666d8f0e8489
I20260812 03:12:19.187604 14969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:19.178984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:49803 (local address 127.26.250.190:39137)
0812 03:12:19.179135 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:19.179139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:19.180655 (+ 1516us) server_negotiation.cc:407] Connection header received
0812 03:12:19.182371 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:19.182375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:19.182440 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:19.182549 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:19.183573 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.184128 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.185374 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.185609 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.185741 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:19.187068 (+ 1327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:19.187071 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:19.187074 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:19.187089 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:19.187117 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:19.187125 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:19.187202 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:19.187374 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 03:12:19.187447 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 03:12:19.188520 15412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:19.179513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:39137 (local address 127.26.250.131:49803)
0812 03:12:19.180466 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:19.180534 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:19.182082 (+ 1548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:19.182749 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:19.182756 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:19.182856 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:19.183424 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:19.183437 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.184279 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.184282 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:19.185219 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:19.185229 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.185819 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.185826 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:19.186001 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:19.186748 (+ 747us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:12:19.186749 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:19.186792 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:12:19.186891 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:19.187217 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:19.187230 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:19.187241 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 03:12:19.187371 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 03:12:19.187681 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":182,"threads_started":1}
I20260812 03:12:19.189762 15410 heartbeater.cc:347] Connected to a master server at 127.26.250.190:39137
I20260812 03:12:19.190044 15410 heartbeater.cc:464] Registering TS with master...
I20260812 03:12:19.190649 15410 heartbeater.cc:511] Master 127.26.250.190:39137 requested a full tablet report, sending...
I20260812 03:12:19.191777 14930 ts_manager.cc:194] Registered new tserver with Master: 94b72db7370c4b63b255e816f5f8fe57 (127.26.250.131:43971)
I20260812 03:12:19.192438 14930 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.250.131:49803
I20260812 03:12:19.194548 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:12:19.200488 14969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:19.196733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34062 (local address 127.26.250.190:39137)
0812 03:12:19.196865 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:19.196869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:19.197024 (+ 155us) server_negotiation.cc:407] Connection header received
0812 03:12:19.197108 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:19.197110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:19.197148 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:19.197214 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:19.197572 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.198077 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.198900 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.199105 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.199181 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:19.200064 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:19.200066 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:19.200068 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:19.200079 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:19.200095 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:19.200102 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:19.200196 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:19.200320 (+ 124us) server_negotiation.cc:299] Negotiation successful
0812 03:12:19.200382 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 03:12:19.203578 14930 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34062:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:12:19.206666 14930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:12:19.222635 15427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:19.216716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56006 (local address 127.26.250.129:42495)
0812 03:12:19.217183 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:19.217202 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:19.217230 (+ 28us) server_negotiation.cc:407] Connection header received
0812 03:12:19.217441 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:19.217446 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:19.217596 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:19.217768 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:19.218699 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.219907 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.221176 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.221807 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.221963 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:19.222220 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:19.222279 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 03:12:19.222416 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":150,"threads_started":1}
I20260812 03:12:19.223464 15423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:19.215664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59000 (local address 127.26.250.131:43971)
0812 03:12:19.216730 (+ 1066us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:19.216746 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:19.217011 (+ 265us) server_negotiation.cc:407] Connection header received
0812 03:12:19.217266 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:19.217272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:19.217508 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:19.217655 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:19.218193 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.219246 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.221038 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.222174 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.222312 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:19.222551 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:19.223172 (+ 621us) server_negotiation.cc:299] Negotiation successful
0812 03:12:19.223281 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":156,"threads_started":1}
I20260812 03:12:19.223459 15426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:19.216326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51826 (local address 127.26.250.130:34063)
0812 03:12:19.216954 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:19.216967 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:19.216996 (+ 29us) server_negotiation.cc:407] Connection header received
0812 03:12:19.217285 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:19.217290 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:19.217508 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:19.217668 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:19.218007 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.219800 (+ 1793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.221247 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.222192 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.222311 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:19.222524 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:19.223172 (+ 648us) server_negotiation.cc:299] Negotiation successful
0812 03:12:19.223281 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":124,"threads_started":1}
I20260812 03:12:19.223527 15424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:19.216296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:42495 (local address 127.0.0.1:56006)
0812 03:12:19.217157 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:19.217171 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:19.217270 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:19.217850 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:19.217853 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:19.217895 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:19.218540 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:19.218549 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.220043 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.220050 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:19.221038 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:19.221049 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.221948 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.221952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:19.222082 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:19.222124 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 03:12:19.222247 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 03:12:19.223404 (+ 1157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":796,"mutex_wait_us":544,"spinlock_wait_cycles":768,"thread_start_us":89,"threads_started":1}
I20260812 03:12:19.223826 15425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:19.216034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:34063 (local address 127.0.0.1:51826)
0812 03:12:19.216891 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:19.216925 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:19.217062 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:19.217697 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:19.217704 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:19.217732 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:19.217865 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:19.217872 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.219922 (+ 2050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.219925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:19.221127 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:19.221139 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.222603 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.222607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:19.222970 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:19.222994 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 03:12:19.223069 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 03:12:19.223707 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":796,"mutex_wait_us":629,"thread_start_us":140,"threads_started":1}
I20260812 03:12:19.224215 15422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:19.215485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:43971 (local address 127.0.0.1:59000)
0812 03:12:19.216860 (+ 1375us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:19.216919 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:19.217073 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:19.217857 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:19.217860 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:19.217902 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:19.218036 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:19.218045 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.219375 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.219378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:19.220905 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:19.220915 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.222604 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.222607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:19.223019 (+ 412us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:19.223046 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 03:12:19.223128 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 03:12:19.224081 (+ 953us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1022,"mutex_wait_us":1178,"spinlock_wait_cycles":37632,"thread_start_us":80,"threads_started":1}
I20260812 03:12:19.227289 15340 tablet_service.cc:1467] Processing CreateTablet for tablet 8bfe9d1aad574727804245a0d8eb778d (DEFAULT_TABLE table=test-table [id=8798bf9017c042c1a88b90b22732da5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:12:19.227289 15200 tablet_service.cc:1467] Processing CreateTablet for tablet 8bfe9d1aad574727804245a0d8eb778d (DEFAULT_TABLE table=test-table [id=8798bf9017c042c1a88b90b22732da5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:12:19.228543 15200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bfe9d1aad574727804245a0d8eb778d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:19.228543 15340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bfe9d1aad574727804245a0d8eb778d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:19.231069 15058 tablet_service.cc:1467] Processing CreateTablet for tablet 8bfe9d1aad574727804245a0d8eb778d (DEFAULT_TABLE table=test-table [id=8798bf9017c042c1a88b90b22732da5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:12:19.232245 15058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8bfe9d1aad574727804245a0d8eb778d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:19.239146 15429 tablet_bootstrap.cc:492] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57: Bootstrap starting.
I20260812 03:12:19.239117 15428 tablet_bootstrap.cc:492] T 8bfe9d1aad574727804245a0d8eb778d P 74bb6887393640dbab379f9af772967f: Bootstrap starting.
I20260812 03:12:19.239984 15430 tablet_bootstrap.cc:492] T 8bfe9d1aad574727804245a0d8eb778d P bc4c52bc738c4c67bd306febb6186c44: Bootstrap starting.
I20260812 03:12:19.241457 15429 tablet_bootstrap.cc:654] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:19.241814 15428 tablet_bootstrap.cc:654] T 8bfe9d1aad574727804245a0d8eb778d P 74bb6887393640dbab379f9af772967f: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:19.242292 15429 log.cc:824] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:19.242545 15430 tablet_bootstrap.cc:654] T 8bfe9d1aad574727804245a0d8eb778d P bc4c52bc738c4c67bd306febb6186c44: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:19.242861 15428 log.cc:824] T 8bfe9d1aad574727804245a0d8eb778d P 74bb6887393640dbab379f9af772967f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:19.243449 15430 log.cc:824] T 8bfe9d1aad574727804245a0d8eb778d P bc4c52bc738c4c67bd306febb6186c44: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:19.244635 15429 tablet_bootstrap.cc:492] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57: No bootstrap required, opened a new log
I20260812 03:12:19.244865 15429 ts_tablet_manager.cc:1397] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:12:19.245095 15428 tablet_bootstrap.cc:492] T 8bfe9d1aad574727804245a0d8eb778d P 74bb6887393640dbab379f9af772967f: No bootstrap required, opened a new log
I20260812 03:12:19.245347 15428 ts_tablet_manager.cc:1397] T 8bfe9d1aad574727804245a0d8eb778d P 74bb6887393640dbab379f9af772967f: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260812 03:12:19.247294 15430 tablet_bootstrap.cc:492] T 8bfe9d1aad574727804245a0d8eb778d P bc4c52bc738c4c67bd306febb6186c44: No bootstrap required, opened a new log
I20260812 03:12:19.247500 15430 ts_tablet_manager.cc:1397] T 8bfe9d1aad574727804245a0d8eb778d P bc4c52bc738c4c67bd306febb6186c44: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20260812 03:12:19.248900 15429 raft_consensus.cc:348] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94b72db7370c4b63b255e816f5f8fe57" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 43971 } } peers { permanent_uuid: "bc4c52bc738c4c67bd306febb6186c44" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 42495 } } peers { permanent_uuid: "74bb6887393640dbab379f9af772967f" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 34063 } }
I20260812 03:12:19.249221 15429 raft_consensus.cc:374] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:19.249289 15429 raft_consensus.cc:729] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94b72db7370c4b63b255e816f5f8fe57, State: Initialized, Role: FOLLOWER
I20260812 03:12:19.249827 15429 consensus_queue.cc:260] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94b72db7370c4b63b255e816f5f8fe57" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 43971 } } peers { permanent_uuid: "bc4c52bc738c4c67bd306febb6186c44" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 42495 } } peers { permanent_uuid: "74bb6887393640dbab379f9af772967f" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 34063 } }
I20260812 03:12:19.250567 15410 heartbeater.cc:503] Master 127.26.250.190:39137 was elected leader, sending a full tablet report...
I20260812 03:12:19.250398 15428 raft_consensus.cc:348] T 8bfe9d1aad574727804245a0d8eb778d P 74bb6887393640dbab379f9af772967f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94b72db7370c4b63b255e816f5f8fe57" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 43971 } } peers { permanent_uuid: "bc4c52bc738c4c67bd306febb6186c44" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 42495 } } peers { permanent_uuid: "74bb6887393640dbab379f9af772967f" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 34063 } }
I20260812 03:12:19.250670 15428 raft_consensus.cc:374] T 8bfe9d1aad574727804245a0d8eb778d P 74bb6887393640dbab379f9af772967f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:19.250725 15428 raft_consensus.cc:729] T 8bfe9d1aad574727804245a0d8eb778d P 74bb6887393640dbab379f9af772967f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74bb6887393640dbab379f9af772967f, State: Initialized, Role: FOLLOWER
I20260812 03:12:19.251044 15429 ts_tablet_manager.cc:1428] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260812 03:12:19.251222 15428 consensus_queue.cc:260] T 8bfe9d1aad574727804245a0d8eb778d P 74bb6887393640dbab379f9af772967f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94b72db7370c4b63b255e816f5f8fe57" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 43971 } } peers { permanent_uuid: "bc4c52bc738c4c67bd306febb6186c44" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 42495 } } peers { permanent_uuid: "74bb6887393640dbab379f9af772967f" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 34063 } }
I20260812 03:12:19.251978 15269 heartbeater.cc:503] Master 127.26.250.190:39137 was elected leader, sending a full tablet report...
I20260812 03:12:19.251899 15430 raft_consensus.cc:348] T 8bfe9d1aad574727804245a0d8eb778d P bc4c52bc738c4c67bd306febb6186c44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94b72db7370c4b63b255e816f5f8fe57" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 43971 } } peers { permanent_uuid: "bc4c52bc738c4c67bd306febb6186c44" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 42495 } } peers { permanent_uuid: "74bb6887393640dbab379f9af772967f" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 34063 } }
I20260812 03:12:19.252148 15430 raft_consensus.cc:374] T 8bfe9d1aad574727804245a0d8eb778d P bc4c52bc738c4c67bd306febb6186c44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:19.252203 15430 raft_consensus.cc:729] T 8bfe9d1aad574727804245a0d8eb778d P bc4c52bc738c4c67bd306febb6186c44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc4c52bc738c4c67bd306febb6186c44, State: Initialized, Role: FOLLOWER
I20260812 03:12:19.252305 15428 ts_tablet_manager.cc:1428] T 8bfe9d1aad574727804245a0d8eb778d P 74bb6887393640dbab379f9af772967f: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
I20260812 03:12:19.252696 15430 consensus_queue.cc:260] T 8bfe9d1aad574727804245a0d8eb778d P bc4c52bc738c4c67bd306febb6186c44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94b72db7370c4b63b255e816f5f8fe57" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 43971 } } peers { permanent_uuid: "bc4c52bc738c4c67bd306febb6186c44" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 42495 } } peers { permanent_uuid: "74bb6887393640dbab379f9af772967f" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 34063 } }
I20260812 03:12:19.253473 15128 heartbeater.cc:503] Master 127.26.250.190:39137 was elected leader, sending a full tablet report...
I20260812 03:12:19.255438 15430 ts_tablet_manager.cc:1428] T 8bfe9d1aad574727804245a0d8eb778d P bc4c52bc738c4c67bd306febb6186c44: Time spent starting tablet: real 0.008s user 0.004s sys 0.003s
W20260812 03:12:19.269069 15129 tablet.cc:2369] T 8bfe9d1aad574727804245a0d8eb778d P bc4c52bc738c4c67bd306febb6186c44: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:12:19.407600 15434 raft_consensus.cc:482] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:12:19.407897 15434 raft_consensus.cc:504] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94b72db7370c4b63b255e816f5f8fe57" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 43971 } } peers { permanent_uuid: "bc4c52bc738c4c67bd306febb6186c44" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 42495 } } peers { permanent_uuid: "74bb6887393640dbab379f9af772967f" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 34063 } }
I20260812 03:12:19.409155 15434 leader_election.cc:288] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc4c52bc738c4c67bd306febb6186c44 (127.26.250.129:42495), 74bb6887393640dbab379f9af772967f (127.26.250.130:34063)
I20260812 03:12:19.414551 15427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:19.409675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:37133 (local address 127.26.250.129:42495)
0812 03:12:19.409799 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:19.409804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:19.410156 (+ 352us) server_negotiation.cc:407] Connection header received
0812 03:12:19.410249 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:19.410251 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:19.410299 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:19.410460 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:19.410712 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.411508 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.413043 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.413805 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.413927 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:19.413971 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:19.414285 (+ 314us) server_negotiation.cc:299] Negotiation successful
0812 03:12:19.414445 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:12:19.414533 15426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:19.409614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:38539 (local address 127.26.250.130:34063)
0812 03:12:19.409824 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:19.409829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:19.409849 (+ 20us) server_negotiation.cc:407] Connection header received
0812 03:12:19.409912 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:19.409917 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:19.409978 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:19.410070 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:19.410399 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.411311 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.413043 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.413744 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.413910 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:19.413967 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:19.414285 (+ 318us) server_negotiation.cc:299] Negotiation successful
0812 03:12:19.414397 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260812 03:12:19.414537 15412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:19.409436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:34063 (local address 127.26.250.131:38539)
0812 03:12:19.409690 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:19.409706 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:19.409799 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:19.410101 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:19.410105 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:19.410123 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:19.410264 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:19.410275 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.411441 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.411445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:19.412880 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:19.412891 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.413912 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.413916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:19.414098 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:19.414141 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 03:12:19.414223 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 03:12:19.414356 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":154}
I20260812 03:12:19.414546 15437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:19.409565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:42495 (local address 127.26.250.131:37133)
0812 03:12:19.410009 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:19.410020 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:19.410141 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:19.410466 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:19.410468 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:19.410484 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:19.410591 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:19.410594 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.411602 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.411605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:19.412879 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:19.412891 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.413912 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.413916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:19.414094 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:19.414138 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 03:12:19.414223 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 03:12:19.414356 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":78,"spinlock_wait_cycles":768,"thread_start_us":195,"threads_started":1}
I20260812 03:12:19.415313 15221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bfe9d1aad574727804245a0d8eb778d" candidate_uuid: "94b72db7370c4b63b255e816f5f8fe57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74bb6887393640dbab379f9af772967f" is_pre_election: true
I20260812 03:12:19.415299 15079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bfe9d1aad574727804245a0d8eb778d" candidate_uuid: "94b72db7370c4b63b255e816f5f8fe57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc4c52bc738c4c67bd306febb6186c44" is_pre_election: true
I20260812 03:12:19.415660 15221 raft_consensus.cc:2454] T 8bfe9d1aad574727804245a0d8eb778d P 74bb6887393640dbab379f9af772967f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94b72db7370c4b63b255e816f5f8fe57 in term 0.
I20260812 03:12:19.415668 15079 raft_consensus.cc:2454] T 8bfe9d1aad574727804245a0d8eb778d P bc4c52bc738c4c67bd306febb6186c44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94b72db7370c4b63b255e816f5f8fe57 in term 0.
I20260812 03:12:19.416234 15293 leader_election.cc:302] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 74bb6887393640dbab379f9af772967f, 94b72db7370c4b63b255e816f5f8fe57, bc4c52bc738c4c67bd306febb6186c44; no voters:
I20260812 03:12:19.416556 15434 raft_consensus.cc:2781] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:12:19.416652 15434 raft_consensus.cc:482] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:12:19.416695 15434 raft_consensus.cc:3037] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:19.417771 15434 raft_consensus.cc:504] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94b72db7370c4b63b255e816f5f8fe57" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 43971 } } peers { permanent_uuid: "bc4c52bc738c4c67bd306febb6186c44" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 42495 } } peers { permanent_uuid: "74bb6887393640dbab379f9af772967f" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 34063 } }
I20260812 03:12:19.418146 15434 leader_election.cc:288] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57 [CANDIDATE]: Term 1 election: Requested vote from peers bc4c52bc738c4c67bd306febb6186c44 (127.26.250.129:42495), 74bb6887393640dbab379f9af772967f (127.26.250.130:34063)
I20260812 03:12:19.418495 15079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bfe9d1aad574727804245a0d8eb778d" candidate_uuid: "94b72db7370c4b63b255e816f5f8fe57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc4c52bc738c4c67bd306febb6186c44"
I20260812 03:12:19.418637 15079 raft_consensus.cc:3037] T 8bfe9d1aad574727804245a0d8eb778d P bc4c52bc738c4c67bd306febb6186c44 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:19.418678 15221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8bfe9d1aad574727804245a0d8eb778d" candidate_uuid: "94b72db7370c4b63b255e816f5f8fe57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74bb6887393640dbab379f9af772967f"
I20260812 03:12:19.418778 15221 raft_consensus.cc:3037] T 8bfe9d1aad574727804245a0d8eb778d P 74bb6887393640dbab379f9af772967f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:19.419768 15221 raft_consensus.cc:2454] T 8bfe9d1aad574727804245a0d8eb778d P 74bb6887393640dbab379f9af772967f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94b72db7370c4b63b255e816f5f8fe57 in term 1.
I20260812 03:12:19.419770 15079 raft_consensus.cc:2454] T 8bfe9d1aad574727804245a0d8eb778d P bc4c52bc738c4c67bd306febb6186c44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94b72db7370c4b63b255e816f5f8fe57 in term 1.
I20260812 03:12:19.420097 15290 leader_election.cc:302] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74bb6887393640dbab379f9af772967f, 94b72db7370c4b63b255e816f5f8fe57; no voters:
I20260812 03:12:19.420282 15434 raft_consensus.cc:2781] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:19.420529 15434 raft_consensus.cc:686] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57 [term 1 LEADER]: Becoming Leader. State: Replica: 94b72db7370c4b63b255e816f5f8fe57, State: Running, Role: LEADER
I20260812 03:12:19.420881 15434 consensus_queue.cc:237] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94b72db7370c4b63b255e816f5f8fe57" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 43971 } } peers { permanent_uuid: "bc4c52bc738c4c67bd306febb6186c44" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 42495 } } peers { permanent_uuid: "74bb6887393640dbab379f9af772967f" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 34063 } }
I20260812 03:12:19.424367 14928 catalog_manager.cc:5495] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94b72db7370c4b63b255e816f5f8fe57 (127.26.250.131). New cstate: current_term: 1 leader_uuid: "94b72db7370c4b63b255e816f5f8fe57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94b72db7370c4b63b255e816f5f8fe57" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 43971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc4c52bc738c4c67bd306febb6186c44" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 42495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74bb6887393640dbab379f9af772967f" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 34063 } health_report { overall_health: UNKNOWN } } }
W20260812 03:12:19.427846 15411 tablet.cc:2369] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:12:19.463955 15423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:19.460195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59002 (local address 127.26.250.131:43971)
0812 03:12:19.460329 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:19.460368 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:19.460382 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:12:19.460457 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:19.460459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:19.460521 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:19.460615 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:12:19.460962 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.461495 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.462473 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:19.462705 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:19.462786 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:19.463133 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:12:19.463564 (+ 431us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:12:19.463674 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:19.463803 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 03:12:19.463837 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 03:12:19.470165 15221 raft_consensus.cc:1261] T 8bfe9d1aad574727804245a0d8eb778d P 74bb6887393640dbab379f9af772967f [term 1 FOLLOWER]: Refusing update from remote peer 94b72db7370c4b63b255e816f5f8fe57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:12:19.470156 15079 raft_consensus.cc:1261] T 8bfe9d1aad574727804245a0d8eb778d P bc4c52bc738c4c67bd306febb6186c44 [term 1 FOLLOWER]: Refusing update from remote peer 94b72db7370c4b63b255e816f5f8fe57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:12:19.470992 15434 consensus_queue.cc:1035] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74bb6887393640dbab379f9af772967f" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 34063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:12:19.471232 15438 consensus_queue.cc:1035] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc4c52bc738c4c67bd306febb6186c44" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 42495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 03:12:19.478647 15270 tablet.cc:2369] T 8bfe9d1aad574727804245a0d8eb778d P 74bb6887393640dbab379f9af772967f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:12:19.478783 15441 mvcc.cc:205] Tried to move back new op lower bound from 7317521774462177280 to 7317521774270795776. Current Snapshot: MvccSnapshot[applied={T|T < 7317521774462177280}]
I20260812 03:12:19.479631 15442 mvcc.cc:205] Tried to move back new op lower bound from 7317521774462177280 to 7317521774270795776. Current Snapshot: MvccSnapshot[applied={T|T < 7317521774462177280}]
I20260812 03:12:19.486161 15443 mvcc.cc:205] Tried to move back new op lower bound from 7317521774462177280 to 7317521774270795776. Current Snapshot: MvccSnapshot[applied={T|T < 7317521774462177280}]
I20260812 03:12:19.489452 14928 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:12:19.489607 14928 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:12:19.492139 14928 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 972eb1d4ac37490797798f4cb13e8bf3: Sending DeleteTablet for 3 replicas of tablet 8bfe9d1aad574727804245a0d8eb778d
I20260812 03:12:19.493119 15058 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bfe9d1aad574727804245a0d8eb778d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:12:19 UTC) from {username='slave'} at 127.0.0.1:56006
I20260812 03:12:19.493299 15340 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bfe9d1aad574727804245a0d8eb778d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:12:19 UTC) from {username='slave'} at 127.0.0.1:59000
I20260812 03:12:19.493902 15200 tablet_service.cc:1514] Processing DeleteTablet for tablet 8bfe9d1aad574727804245a0d8eb778d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:12:19 UTC) from {username='slave'} at 127.0.0.1:51826
I20260812 03:12:19.494000 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 14992
I20260812 03:12:19.494917 15454 tablet_replica.cc:330] T 8bfe9d1aad574727804245a0d8eb778d P 74bb6887393640dbab379f9af772967f: stopping tablet replica
I20260812 03:12:19.495252 15454 raft_consensus.cc:2229] T 8bfe9d1aad574727804245a0d8eb778d P 74bb6887393640dbab379f9af772967f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:12:19.495604 15454 raft_consensus.cc:2258] T 8bfe9d1aad574727804245a0d8eb778d P 74bb6887393640dbab379f9af772967f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:12:19.497162 15453 tablet_replica.cc:330] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57: stopping tablet replica
I20260812 03:12:19.497485 15453 raft_consensus.cc:2229] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57 [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:12:19.497504 15454 ts_tablet_manager.cc:1905] T 8bfe9d1aad574727804245a0d8eb778d P 74bb6887393640dbab379f9af772967f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:12:19.497918 15453 raft_consensus.cc:2258] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:12:19.499437 15453 ts_tablet_manager.cc:1905] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:12:19.500792 15454 ts_tablet_manager.cc:1918] T 8bfe9d1aad574727804245a0d8eb778d P 74bb6887393640dbab379f9af772967f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:12:19.500913 15454 log.cc:1196] T 8bfe9d1aad574727804245a0d8eb778d P 74bb6887393640dbab379f9af772967f: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-1/wal/wals/8bfe9d1aad574727804245a0d8eb778d
I20260812 03:12:19.501282 15454 ts_tablet_manager.cc:1939] T 8bfe9d1aad574727804245a0d8eb778d P 74bb6887393640dbab379f9af772967f: Deleting consensus metadata
I20260812 03:12:19.501878 15453 ts_tablet_manager.cc:1918] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:12:19.501998 15453 log.cc:1196] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/8bfe9d1aad574727804245a0d8eb778d
I20260812 03:12:19.502278 15453 ts_tablet_manager.cc:1939] T 8bfe9d1aad574727804245a0d8eb778d P 94b72db7370c4b63b255e816f5f8fe57: Deleting consensus metadata
I20260812 03:12:19.502974 14911 catalog_manager.cc:4841] TS 74bb6887393640dbab379f9af772967f (127.26.250.130:34063): tablet 8bfe9d1aad574727804245a0d8eb778d (table test-table [id=8798bf9017c042c1a88b90b22732da5f]) successfully deleted
I20260812 03:12:19.503533 14914 catalog_manager.cc:4841] TS 94b72db7370c4b63b255e816f5f8fe57 (127.26.250.131:43971): tablet 8bfe9d1aad574727804245a0d8eb778d (table test-table [id=8798bf9017c042c1a88b90b22732da5f]) successfully deleted
I20260812 03:12:19.510457 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 15131
W20260812 03:12:19.510468 14914 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.250.129:42495 (error 108)
I20260812 03:12:19.512295 15422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:19.511902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:42495 (local address 127.0.0.1:56018)
0812 03:12:19.512068 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:19.512175 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.129:42495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 03:12:19.512811 14914 catalog_manager.cc:4584] TS bc4c52bc738c4c67bd306febb6186c44 (127.26.250.129:42495): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8bfe9d1aad574727804245a0d8eb778d: Network error: Client connection negotiation failed: client connection to 127.26.250.129:42495: connect: Connection refused (error 111)
I20260812 03:12:19.519676 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 15272
I20260812 03:12:19.527490 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 14896
2026-08-12T03:12:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1291 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:12:19 dist-test-slave-zpfg krb5kdc[15458](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:12:19 dist-test-slave-zpfg krb5kdc[15458](info): set up 2 sockets
Aug 12 03:12:19 dist-test-slave-zpfg krb5kdc[15458](info): commencing operation
krb5kdc: starting...
W20260812 03:12:20.647866 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:12:20 dist-test-slave-zpfg krb5kdc[15458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:12:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:12:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:12:20.793346 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:40095
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:45923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:40095
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:20.921111 15471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:20.921439 15471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:20.921483 15471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:20.925922 15471 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:20.926020 15471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:20.926043 15471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:20.926060 15471 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:20.926076 15471 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:20.932272 15471 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:45923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:40095
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:40095
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:20.933550 15471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:20.934751 15471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:20.944414 15480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:20.944479 15478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:20.944505 15476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:20.944631 15471 server_base.cc:1034] running on GCE node
I20260812 03:12:20.945659 15471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:20.946993 15471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:20.948199 15471 hybrid_clock.cc:648] HybridClock initialized: now 1786504340948182 us; error 35 us; skew 500 ppm
Aug 12 03:12:20 dist-test-slave-zpfg krb5kdc[15458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504340, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:20.960420 15471 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:12:20.961531 15471 webserver.cc:460] Webserver started at http://127.26.250.190:34615/ using document root <none> and password file <none>
I20260812 03:12:20.962199 15471 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:20.962301 15471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:20.962639 15471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:20.964947 15471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "69cf972ff98647c4b44aba4028cb6a3f"
format_stamp: "Formatted at 2026-08-12 03:12:20 on dist-test-slave-zpfg"
server_key: "50cf9dc2c9b546cbc070e4cde147690d"
server_key_iv: "22970103406d913949cadccf217c93db"
server_key_version: "encryptionkey@0"
I20260812 03:12:20.965585 15471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "69cf972ff98647c4b44aba4028cb6a3f"
format_stamp: "Formatted at 2026-08-12 03:12:20 on dist-test-slave-zpfg"
server_key: "50cf9dc2c9b546cbc070e4cde147690d"
server_key_iv: "22970103406d913949cadccf217c93db"
server_key_version: "encryptionkey@0"
I20260812 03:12:20.969811 15471 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260812 03:12:20.972756 15487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:20.974802 15471 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:12:20.974967 15471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "69cf972ff98647c4b44aba4028cb6a3f"
format_stamp: "Formatted at 2026-08-12 03:12:20 on dist-test-slave-zpfg"
server_key: "50cf9dc2c9b546cbc070e4cde147690d"
server_key_iv: "22970103406d913949cadccf217c93db"
server_key_version: "encryptionkey@0"
I20260812 03:12:20.975108 15471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:21.003103 15471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:21.006789 15471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:21.007042 15471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:21.014894 15471 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:40095
I20260812 03:12:21.014945 15544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:40095 every 8 connection(s)
I20260812 03:12:21.015980 15471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:12:21.019106 15545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:21.022802 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 15471
I20260812 03:12:21.022972 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:12:21.023273 27626 external_mini_cluster.cc:1428] Setting key 7ae5b7e8e39f6ce1ea5acee7cb6d4327
I20260812 03:12:21.025564 15545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69cf972ff98647c4b44aba4028cb6a3f: Bootstrap starting.
I20260812 03:12:21.028450 15545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69cf972ff98647c4b44aba4028cb6a3f: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:21.029405 15545 log.cc:824] T 00000000000000000000000000000000 P 69cf972ff98647c4b44aba4028cb6a3f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:21.032019 15545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69cf972ff98647c4b44aba4028cb6a3f: No bootstrap required, opened a new log
Aug 12 03:12:21 dist-test-slave-zpfg krb5kdc[15458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:12:21.036608 15545 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69cf972ff98647c4b44aba4028cb6a3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69cf972ff98647c4b44aba4028cb6a3f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 40095 } }
I20260812 03:12:21.036972 15545 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69cf972ff98647c4b44aba4028cb6a3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:21.037055 15545 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69cf972ff98647c4b44aba4028cb6a3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69cf972ff98647c4b44aba4028cb6a3f, State: Initialized, Role: FOLLOWER
I20260812 03:12:21.037611 15545 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69cf972ff98647c4b44aba4028cb6a3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69cf972ff98647c4b44aba4028cb6a3f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 40095 } }
I20260812 03:12:21.037801 15545 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69cf972ff98647c4b44aba4028cb6a3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:12:21.037894 15545 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69cf972ff98647c4b44aba4028cb6a3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:12:21.037994 15545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69cf972ff98647c4b44aba4028cb6a3f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:21.039319 15545 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69cf972ff98647c4b44aba4028cb6a3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69cf972ff98647c4b44aba4028cb6a3f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 40095 } }
I20260812 03:12:21.039713 15545 leader_election.cc:302] T 00000000000000000000000000000000 P 69cf972ff98647c4b44aba4028cb6a3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69cf972ff98647c4b44aba4028cb6a3f; no voters:
I20260812 03:12:21.040086 15545 leader_election.cc:288] T 00000000000000000000000000000000 P 69cf972ff98647c4b44aba4028cb6a3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:12:21.040259 15550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69cf972ff98647c4b44aba4028cb6a3f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:21.040360 15548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:21.024720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50342 (local address 127.26.250.190:40095)
0812 03:12:21.025163 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:21.025180 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:21.025218 (+ 38us) server_negotiation.cc:407] Connection header received
0812 03:12:21.025938 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:21.025958 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:21.026310 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:21.026677 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:21.027687 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.028781 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.029969 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.030675 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.030795 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:21.033750 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:21.033757 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:21.033779 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:21.033816 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:21.036745 (+ 2929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:21.037337 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:21.037344 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:21.037350 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:21.037500 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:21.037777 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:21.037780 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:21.037782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:21.038445 (+ 663us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:21.038643 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:21.038926 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 03:12:21.039277 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":135,"threads_started":1}
I20260812 03:12:21.040575 15550 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69cf972ff98647c4b44aba4028cb6a3f [term 1 LEADER]: Becoming Leader. State: Replica: 69cf972ff98647c4b44aba4028cb6a3f, State: Running, Role: LEADER
I20260812 03:12:21.040896 15550 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69cf972ff98647c4b44aba4028cb6a3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69cf972ff98647c4b44aba4028cb6a3f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 40095 } }
I20260812 03:12:21.041347 15545 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69cf972ff98647c4b44aba4028cb6a3f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:21.043471 15553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69cf972ff98647c4b44aba4028cb6a3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69cf972ff98647c4b44aba4028cb6a3f. Latest consensus state: current_term: 1 leader_uuid: "69cf972ff98647c4b44aba4028cb6a3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69cf972ff98647c4b44aba4028cb6a3f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 40095 } } }
I20260812 03:12:21.043697 15551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69cf972ff98647c4b44aba4028cb6a3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69cf972ff98647c4b44aba4028cb6a3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69cf972ff98647c4b44aba4028cb6a3f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 40095 } } }
I20260812 03:12:21.044296 15553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69cf972ff98647c4b44aba4028cb6a3f [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:21.044381 15551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69cf972ff98647c4b44aba4028cb6a3f [sys.catalog]: This master's current role is: LEADER
W20260812 03:12:21.046481 15566 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 69cf972ff98647c4b44aba4028cb6a3f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:12:21.046550 15566 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:12:21.046648 15568 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:12:21.047673 15568 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:12:21.053881 15568 catalog_manager.cc:1347] Generated new cluster ID: 83551ea37588462f9e75a2c1a7401eb7
I20260812 03:12:21.053956 15568 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:12:21.074961 15568 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:12:21.076020 15568 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:12:21.091676 15568 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69cf972ff98647c4b44aba4028cb6a3f: Generated new TSK 0
I20260812 03:12:21.092502 15568 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.26.250.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
Entry for principal kudu/127.26.250.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab.
I20260812 03:12:21.131669 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:0
--local_ip_for_outbound_sockets=127.26.250.129
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:40095
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:45923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:21.268680 15573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:21.268985 15573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:21.269033 15573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:21.273384 15573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:21.273473 15573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:21.273569 15573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:12:21.279582 15573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:45923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.129.keytab
--principal=kudu/127.26.250.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:40095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:21.280831 15573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:21.281980 15573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:21.292961 15578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:21.292961 15579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:21.292987 15581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:21.293292 15573 server_base.cc:1034] running on GCE node
I20260812 03:12:21.293812 15573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:21.294602 15573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:21.295786 15573 hybrid_clock.cc:648] HybridClock initialized: now 1786504341295768 us; error 39 us; skew 500 ppm
Aug 12 03:12:21 dist-test-slave-zpfg krb5kdc[15458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504341, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:21.298945 15573 init.cc:377] Logged in from keytab as kudu/127.26.250.129@KRBTEST.COM (short username kudu)
I20260812 03:12:21.299998 15573 webserver.cc:460] Webserver started at http://127.26.250.129:33499/ using document root <none> and password file <none>
I20260812 03:12:21.300655 15573 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:21.300755 15573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:21.301014 15573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:21.303419 15573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-0/data/instance:
uuid: "f7feab4f02f74c138c5b704ed5c89297"
format_stamp: "Formatted at 2026-08-12 03:12:21 on dist-test-slave-zpfg"
server_key: "38cf03fc7173fe551814685cf54893f1"
server_key_iv: "8a69ad40e35438a98eeca096e1645eb1"
server_key_version: "encryptionkey@0"
I20260812 03:12:21.304026 15573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance:
uuid: "f7feab4f02f74c138c5b704ed5c89297"
format_stamp: "Formatted at 2026-08-12 03:12:21 on dist-test-slave-zpfg"
server_key: "38cf03fc7173fe551814685cf54893f1"
server_key_iv: "8a69ad40e35438a98eeca096e1645eb1"
server_key_version: "encryptionkey@0"
I20260812 03:12:21.308111 15573 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:12:21.311111 15588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:21.312552 15573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:12:21.312714 15573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "f7feab4f02f74c138c5b704ed5c89297"
format_stamp: "Formatted at 2026-08-12 03:12:21 on dist-test-slave-zpfg"
server_key: "38cf03fc7173fe551814685cf54893f1"
server_key_iv: "8a69ad40e35438a98eeca096e1645eb1"
server_key_version: "encryptionkey@0"
I20260812 03:12:21.312844 15573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:21.330317 15573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:21.333814 15573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:21.334046 15573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:21.334762 15573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:12:21.335664 15573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:12:21.335736 15573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:21.335829 15573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:12:21.335875 15573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:21.347204 15573 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:45631
I20260812 03:12:21.347313 15709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:45631 every 8 connection(s)
I20260812 03:12:21.348270 15573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:12:21.351326 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 15573
I20260812 03:12:21.351480 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance
I20260812 03:12:21.351826 27626 external_mini_cluster.cc:1428] Setting key 12e529d65b59d47f323e4276df62b9db
Aug 12 03:12:21 dist-test-slave-zpfg krb5kdc[15458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504341, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.129@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.250.130@KRBTEST.COM; defaulting to no policy
I20260812 03:12:21.365561 15548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:21.350552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:50877 (local address 127.26.250.190:40095)
0812 03:12:21.350747 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:21.350753 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:21.351743 (+ 990us) server_negotiation.cc:407] Connection header received
0812 03:12:21.353612 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:21.353616 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:21.353675 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:21.353793 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:21.355232 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.356271 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.357547 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.358016 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.358159 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:21.360982 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:21.360988 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:21.360995 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:21.361030 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:21.363050 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:21.363624 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:21.363629 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:21.363633 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:21.363746 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:21.364198 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:21.364201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:21.364203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:21.364419 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:21.364510 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:21.365263 (+ 753us) server_negotiation.cc:299] Negotiation successful
0812 03:12:21.365427 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 03:12:21.366638 15712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:21.350872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:40095 (local address 127.26.250.129:50877)
0812 03:12:21.351543 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:21.351611 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:21.353315 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:21.354001 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:21.354009 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:21.354493 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:21.355070 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:21.355085 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.356447 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.356450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:21.357401 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:21.357411 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.358190 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.358193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:21.358422 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:21.359192 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:21.359210 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:21.360775 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:21.363188 (+ 2413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:21.363193 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:21.363208 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:21.363491 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:21.363895 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:21.363898 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:21.363900 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:21.364063 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:21.364555 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:21.364560 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:21.364882 (+ 322us) client_negotiation.cc:769] Sending connection context
0812 03:12:21.365193 (+ 311us) client_negotiation.cc:240] Negotiation successful
0812 03:12:21.365528 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":169,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.130@KRBTEST.COM" created.
I20260812 03:12:21.368196 15710 heartbeater.cc:347] Connected to a master server at 127.26.250.190:40095
I20260812 03:12:21.368484 15710 heartbeater.cc:464] Registering TS with master...
I20260812 03:12:21.369014 15710 heartbeater.cc:511] Master 127.26.250.190:40095 requested a full tablet report, sending...
I20260812 03:12:21.370751 15507 ts_manager.cc:194] Registered new tserver with Master: f7feab4f02f74c138c5b704ed5c89297 (127.26.250.129:45631)
I20260812 03:12:21.371966 15507 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.129@KRBTEST.COM'} at 127.26.250.129:50877
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
Entry for principal kudu/127.26.250.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab.
I20260812 03:12:21.385488 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:0
--local_ip_for_outbound_sockets=127.26.250.130
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:40095
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:45923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:21.504617 15715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:21.504952 15715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:21.505033 15715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:21.509234 15715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:21.509341 15715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:21.509438 15715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:12:21.515076 15715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:45923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.130.keytab
--principal=kudu/127.26.250.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:40095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:21.516237 15715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:21.517326 15715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:21.527441 15723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:21.527446 15721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:21.527446 15720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:21.527667 15715 server_base.cc:1034] running on GCE node
I20260812 03:12:21.528206 15715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:21.528911 15715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:21.530114 15715 hybrid_clock.cc:648] HybridClock initialized: now 1786504341530095 us; error 31 us; skew 500 ppm
Aug 12 03:12:21 dist-test-slave-zpfg krb5kdc[15458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504341, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:21.533432 15715 init.cc:377] Logged in from keytab as kudu/127.26.250.130@KRBTEST.COM (short username kudu)
I20260812 03:12:21.534411 15715 webserver.cc:460] Webserver started at http://127.26.250.130:45829/ using document root <none> and password file <none>
I20260812 03:12:21.534991 15715 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:21.535055 15715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:21.535231 15715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:21.537451 15715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-1/data/instance:
uuid: "791cb07119a6494a87457d8b7a0b9b84"
format_stamp: "Formatted at 2026-08-12 03:12:21 on dist-test-slave-zpfg"
server_key: "204a1734bf4584328e174437d9bd5950"
server_key_iv: "9b2c71e758a6c563d4e5c9ce39c050dd"
server_key_version: "encryptionkey@0"
I20260812 03:12:21.538023 15715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance:
uuid: "791cb07119a6494a87457d8b7a0b9b84"
format_stamp: "Formatted at 2026-08-12 03:12:21 on dist-test-slave-zpfg"
server_key: "204a1734bf4584328e174437d9bd5950"
server_key_iv: "9b2c71e758a6c563d4e5c9ce39c050dd"
server_key_version: "encryptionkey@0"
I20260812 03:12:21.542187 15715 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 03:12:21.545077 15730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:21.546598 15715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:12:21.546702 15715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "791cb07119a6494a87457d8b7a0b9b84"
format_stamp: "Formatted at 2026-08-12 03:12:21 on dist-test-slave-zpfg"
server_key: "204a1734bf4584328e174437d9bd5950"
server_key_iv: "9b2c71e758a6c563d4e5c9ce39c050dd"
server_key_version: "encryptionkey@0"
I20260812 03:12:21.546797 15715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:21.560551 15715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:21.564241 15715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:21.564457 15715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:21.565074 15715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:12:21.565924 15715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:12:21.565972 15715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:21.566013 15715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:12:21.566030 15715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:21.575500 15715 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:37763
I20260812 03:12:21.575529 15860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:37763 every 8 connection(s)
I20260812 03:12:21.576543 15715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:12:21.583287 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 15715
I20260812 03:12:21.583433 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance
I20260812 03:12:21.583755 27626 external_mini_cluster.cc:1428] Setting key 0a603d1e956fae18a43d6e1df397737a
Aug 12 03:12:21 dist-test-slave-zpfg krb5kdc[15458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504341, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.130@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:12:21.593808 15548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:21.578529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:59983 (local address 127.26.250.190:40095)
0812 03:12:21.578695 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:21.578700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:21.579824 (+ 1124us) server_negotiation.cc:407] Connection header received
0812 03:12:21.581306 (+ 1482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:21.581309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:21.581369 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:21.581457 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:21.582883 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.583677 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.585082 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.585358 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.585445 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:21.589346 (+ 3901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:21.589350 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:21.589353 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:21.589387 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:21.591118 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:21.591745 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:21.591749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:21.591751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:21.591852 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:21.592403 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:21.592406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:21.592407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:21.592651 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:21.592736 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:21.593534 (+ 798us) server_negotiation.cc:299] Negotiation successful
0812 03:12:21.593651 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 03:12:21.595292 15863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:21.579028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:40095 (local address 127.26.250.130:59983)
0812 03:12:21.579662 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:21.579711 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:21.580984 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:21.581663 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:21.581675 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:21.582127 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:21.582714 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:21.582728 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.583835 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.583841 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:21.584937 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:21.584951 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.585539 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.585543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:21.585771 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:21.586695 (+ 924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:21.586722 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:21.589110 (+ 2388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:21.591290 (+ 2180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:21.591297 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:21.591316 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:21.591619 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:21.591994 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:21.592000 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:21.592006 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:21.592266 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:21.592777 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:21.592788 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:21.593212 (+ 424us) client_negotiation.cc:769] Sending connection context
0812 03:12:21.593477 (+ 265us) client_negotiation.cc:240] Negotiation successful
0812 03:12:21.593920 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":130,"threads_started":1}
WARNING: no policy specified for kudu/127.26.250.131@KRBTEST.COM; defaulting to no policy
I20260812 03:12:21.596843 15861 heartbeater.cc:347] Connected to a master server at 127.26.250.190:40095
I20260812 03:12:21.597177 15861 heartbeater.cc:464] Registering TS with master...
I20260812 03:12:21.597862 15861 heartbeater.cc:511] Master 127.26.250.190:40095 requested a full tablet report, sending...
I20260812 03:12:21.599376 15507 ts_manager.cc:194] Registered new tserver with Master: 791cb07119a6494a87457d8b7a0b9b84 (127.26.250.130:37763)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.131@KRBTEST.COM" created.
I20260812 03:12:21.600193 15507 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.130@KRBTEST.COM'} at 127.26.250.130:59983
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
Entry for principal kudu/127.26.250.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab.
I20260812 03:12:21.618323 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:0
--local_ip_for_outbound_sockets=127.26.250.131
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:40095
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:45923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:21.740525 15866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:21.740842 15866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:21.740937 15866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:21.745667 15866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:21.745744 15866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:21.745851 15866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:12:21.752005 15866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:45923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.131.keytab
--principal=kudu/127.26.250.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:40095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:21.753196 15866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:21.754295 15866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:21.764683 15872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:21.764683 15874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:21.764881 15866 server_base.cc:1034] running on GCE node
W20260812 03:12:21.764720 15871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:21.765496 15866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:21.766316 15866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:21.767546 15866 hybrid_clock.cc:648] HybridClock initialized: now 1786504341767532 us; error 29 us; skew 500 ppm
Aug 12 03:12:21 dist-test-slave-zpfg krb5kdc[15458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504341, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:21.770634 15866 init.cc:377] Logged in from keytab as kudu/127.26.250.131@KRBTEST.COM (short username kudu)
I20260812 03:12:21.771652 15866 webserver.cc:460] Webserver started at http://127.26.250.131:43793/ using document root <none> and password file <none>
I20260812 03:12:21.772246 15866 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:21.772310 15866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:21.772491 15866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:21.774842 15866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-2/data/instance:
uuid: "808e8ebe19f94e9485fd133fed9a68f3"
format_stamp: "Formatted at 2026-08-12 03:12:21 on dist-test-slave-zpfg"
server_key: "5314c9cb230e46ef99ea88484cfb897e"
server_key_iv: "1db16fe64153d16819004dca9bd4353e"
server_key_version: "encryptionkey@0"
I20260812 03:12:21.775419 15866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance:
uuid: "808e8ebe19f94e9485fd133fed9a68f3"
format_stamp: "Formatted at 2026-08-12 03:12:21 on dist-test-slave-zpfg"
server_key: "5314c9cb230e46ef99ea88484cfb897e"
server_key_iv: "1db16fe64153d16819004dca9bd4353e"
server_key_version: "encryptionkey@0"
I20260812 03:12:21.779721 15866 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:12:21.782536 15883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:21.783885 15866 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:12:21.784019 15866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "808e8ebe19f94e9485fd133fed9a68f3"
format_stamp: "Formatted at 2026-08-12 03:12:21 on dist-test-slave-zpfg"
server_key: "5314c9cb230e46ef99ea88484cfb897e"
server_key_iv: "1db16fe64153d16819004dca9bd4353e"
server_key_version: "encryptionkey@0"
I20260812 03:12:21.784139 15866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:21.805769 15866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:21.809391 15866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:21.809630 15866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:21.810295 15866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:12:21.811280 15866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:12:21.811328 15866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:21.811372 15866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:12:21.811388 15866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:21.821884 15866 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:34667
I20260812 03:12:21.821918 16003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:34667 every 8 connection(s)
I20260812 03:12:21.823004 15866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:12:21.827054 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 15866
I20260812 03:12:21.827420 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance
I20260812 03:12:21.827950 27626 external_mini_cluster.cc:1428] Setting key 793ee3e109246cc5b3c0a26266d1a354
Aug 12 03:12:21 dist-test-slave-zpfg krb5kdc[15458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504341, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.131@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:12:21.839316 15548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:21.825170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:34581 (local address 127.26.250.190:40095)
0812 03:12:21.825328 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:21.825332 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:21.826270 (+ 938us) server_negotiation.cc:407] Connection header received
0812 03:12:21.828022 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:21.828026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:21.828092 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:21.828206 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:21.829865 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.830699 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.831921 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.832188 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.832303 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:21.835294 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:21.835298 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:21.835301 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:21.835330 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:21.837084 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:21.837609 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:21.837612 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:21.837613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:21.837705 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:21.838043 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:21.838045 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:21.838046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:21.838263 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:21.838366 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:21.838984 (+ 618us) server_negotiation.cc:299] Negotiation successful
0812 03:12:21.839110 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 03:12:21.840158 16006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:21.825593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:40095 (local address 127.26.250.131:34581)
0812 03:12:21.826104 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:21.826156 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:21.827778 (+ 1622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:21.828445 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:21.828456 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:21.828923 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:21.829648 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:21.829667 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.830857 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.830860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:21.831788 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:21.831798 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.832371 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.832375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:21.832522 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:21.833153 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:21.833170 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:21.835014 (+ 1844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:21.837196 (+ 2182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:21.837201 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:21.837216 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:21.837489 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:21.837801 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:21.837803 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:21.837805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:21.837950 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:21.838447 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:21.838453 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:21.838717 (+ 264us) client_negotiation.cc:769] Sending connection context
0812 03:12:21.838885 (+ 168us) client_negotiation.cc:240] Negotiation successful
0812 03:12:21.839172 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":130,"threads_started":1}
I20260812 03:12:21.841645 16004 heartbeater.cc:347] Connected to a master server at 127.26.250.190:40095
I20260812 03:12:21.841950 16004 heartbeater.cc:464] Registering TS with master...
I20260812 03:12:21.842641 16004 heartbeater.cc:511] Master 127.26.250.190:40095 requested a full tablet report, sending...
I20260812 03:12:21.843878 15507 ts_manager.cc:194] Registered new tserver with Master: 808e8ebe19f94e9485fd133fed9a68f3 (127.26.250.131:34667)
I20260812 03:12:21.844513 15507 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.26.250.131@KRBTEST.COM'} at 127.26.250.131:34581
I20260812 03:12:21.855544 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:12:21.866848 15548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:21.858026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50356 (local address 127.26.250.190:40095)
0812 03:12:21.858223 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:21.858229 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:21.858448 (+ 219us) server_negotiation.cc:407] Connection header received
0812 03:12:21.858636 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:21.858639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:21.858687 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:21.858784 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:21.859649 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.860255 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.861220 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.861469 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.861591 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:21.863286 (+ 1695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:21.863290 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:21.863292 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:21.863320 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:21.864846 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:21.865337 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:21.865343 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:21.865350 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:21.865434 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:21.865794 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:21.865799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:21.865801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:21.866053 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:21.866224 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:21.866523 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 03:12:21.866651 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 03:12:21.870553 15507 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:12:21.873924 15507 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:12:21.893843 16019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:21.886942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57874 (local address 127.26.250.130:37763)
0812 03:12:21.887885 (+ 943us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:21.887901 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:21.888814 (+ 913us) server_negotiation.cc:407] Connection header received
0812 03:12:21.889010 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:21.889019 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:21.889493 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:21.889680 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:21.889953 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.890950 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.892131 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.893047 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.893204 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:21.893359 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:21.893565 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 03:12:21.893667 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":105,"threads_started":1}
I20260812 03:12:21.893826 16018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:21.886748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:37763 (local address 127.0.0.1:57874)
0812 03:12:21.887005 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:21.887018 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:21.887085 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:21.889650 (+ 2565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:21.889653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:21.889673 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:21.889791 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:21.889799 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.891087 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.891092 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:21.891987 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:21.891995 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.893192 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.893196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:21.893322 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:21.893376 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 03:12:21.893521 (+ 145us) client_negotiation.cc:240] Negotiation successful
0812 03:12:21.893665 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":50,"thread_start_us":110,"threads_started":1}
I20260812 03:12:21.894093 16017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:21.886263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:34667 (local address 127.0.0.1:50654)
0812 03:12:21.886741 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:21.886755 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:21.886838 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:21.888637 (+ 1799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:21.888644 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:21.888670 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:21.888837 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:21.888855 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.890526 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.890530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:21.892215 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:21.892225 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.893506 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.893510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:21.893610 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:21.893638 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 03:12:21.893706 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 03:12:21.893789 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":50,"thread_start_us":137,"threads_started":1}
I20260812 03:12:21.894766 16015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:21.885874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:45631 (local address 127.0.0.1:50506)
0812 03:12:21.886266 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:21.886322 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:21.886485 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:21.888664 (+ 2179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:21.888668 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:21.888690 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:21.888899 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:21.888949 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.891114 (+ 2165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.891118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:21.892742 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:21.892752 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.894369 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.894376 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:21.894474 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:21.894497 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 03:12:21.894554 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 03:12:21.894631 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":241,"thread_start_us":117,"threads_started":1}
I20260812 03:12:21.894907 16020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:21.887481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50654 (local address 127.26.250.131:34667)
0812 03:12:21.888041 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:21.888054 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:21.888078 (+ 24us) server_negotiation.cc:407] Connection header received
0812 03:12:21.888165 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:21.888170 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:21.888433 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:21.889023 (+ 590us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:21.889211 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.890395 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.892365 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.893320 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.894422 (+ 1102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:21.894578 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:21.894633 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 03:12:21.894742 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":139,"threads_started":1}
I20260812 03:12:21.895576 16016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:21.886012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50506 (local address 127.26.250.129:45631)
0812 03:12:21.887178 (+ 1166us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:21.887193 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:21.888081 (+ 888us) server_negotiation.cc:407] Connection header received
0812 03:12:21.888199 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:21.888205 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:21.888441 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:21.888630 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:21.889886 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.890963 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.892891 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.894198 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.895127 (+ 929us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:21.895276 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:21.895328 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 03:12:21.895422 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1016,"thread_start_us":122,"threads_started":1}
I20260812 03:12:21.897294 15783 tablet_service.cc:1467] Processing CreateTablet for tablet f2abbf7df7ed47c1a6938d466999bef6 (DEFAULT_TABLE table=test-table [id=950b500dc76f41178fc1647eb4a8b645]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:12:21.898393 15783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2abbf7df7ed47c1a6938d466999bef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:21.897966 15930 tablet_service.cc:1467] Processing CreateTablet for tablet f2abbf7df7ed47c1a6938d466999bef6 (DEFAULT_TABLE table=test-table [id=950b500dc76f41178fc1647eb4a8b645]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:12:21.898442 15639 tablet_service.cc:1467] Processing CreateTablet for tablet f2abbf7df7ed47c1a6938d466999bef6 (DEFAULT_TABLE table=test-table [id=950b500dc76f41178fc1647eb4a8b645]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:12:21.899200 15930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2abbf7df7ed47c1a6938d466999bef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:21.899600 15639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2abbf7df7ed47c1a6938d466999bef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:21.905686 16021 tablet_bootstrap.cc:492] T f2abbf7df7ed47c1a6938d466999bef6 P 791cb07119a6494a87457d8b7a0b9b84: Bootstrap starting.
I20260812 03:12:21.907418 16022 tablet_bootstrap.cc:492] T f2abbf7df7ed47c1a6938d466999bef6 P f7feab4f02f74c138c5b704ed5c89297: Bootstrap starting.
I20260812 03:12:21.908424 16023 tablet_bootstrap.cc:492] T f2abbf7df7ed47c1a6938d466999bef6 P 808e8ebe19f94e9485fd133fed9a68f3: Bootstrap starting.
I20260812 03:12:21.909675 16022 tablet_bootstrap.cc:654] T f2abbf7df7ed47c1a6938d466999bef6 P f7feab4f02f74c138c5b704ed5c89297: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:21.909677 16021 tablet_bootstrap.cc:654] T f2abbf7df7ed47c1a6938d466999bef6 P 791cb07119a6494a87457d8b7a0b9b84: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:21.910568 16023 tablet_bootstrap.cc:654] T f2abbf7df7ed47c1a6938d466999bef6 P 808e8ebe19f94e9485fd133fed9a68f3: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:21.910748 16022 log.cc:824] T f2abbf7df7ed47c1a6938d466999bef6 P f7feab4f02f74c138c5b704ed5c89297: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:21.911000 16021 log.cc:824] T f2abbf7df7ed47c1a6938d466999bef6 P 791cb07119a6494a87457d8b7a0b9b84: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:21.911957 16023 log.cc:824] T f2abbf7df7ed47c1a6938d466999bef6 P 808e8ebe19f94e9485fd133fed9a68f3: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:21.913244 16021 tablet_bootstrap.cc:492] T f2abbf7df7ed47c1a6938d466999bef6 P 791cb07119a6494a87457d8b7a0b9b84: No bootstrap required, opened a new log
I20260812 03:12:21.913244 16022 tablet_bootstrap.cc:492] T f2abbf7df7ed47c1a6938d466999bef6 P f7feab4f02f74c138c5b704ed5c89297: No bootstrap required, opened a new log
I20260812 03:12:21.913475 16022 ts_tablet_manager.cc:1397] T f2abbf7df7ed47c1a6938d466999bef6 P f7feab4f02f74c138c5b704ed5c89297: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260812 03:12:21.913475 16021 ts_tablet_manager.cc:1397] T f2abbf7df7ed47c1a6938d466999bef6 P 791cb07119a6494a87457d8b7a0b9b84: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260812 03:12:21.914037 16023 tablet_bootstrap.cc:492] T f2abbf7df7ed47c1a6938d466999bef6 P 808e8ebe19f94e9485fd133fed9a68f3: No bootstrap required, opened a new log
I20260812 03:12:21.914271 16023 ts_tablet_manager.cc:1397] T f2abbf7df7ed47c1a6938d466999bef6 P 808e8ebe19f94e9485fd133fed9a68f3: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 03:12:21.917991 16021 raft_consensus.cc:348] T f2abbf7df7ed47c1a6938d466999bef6 P 791cb07119a6494a87457d8b7a0b9b84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7feab4f02f74c138c5b704ed5c89297" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 45631 } } peers { permanent_uuid: "791cb07119a6494a87457d8b7a0b9b84" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 37763 } } peers { permanent_uuid: "808e8ebe19f94e9485fd133fed9a68f3" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 34667 } }
I20260812 03:12:21.917994 16023 raft_consensus.cc:348] T f2abbf7df7ed47c1a6938d466999bef6 P 808e8ebe19f94e9485fd133fed9a68f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7feab4f02f74c138c5b704ed5c89297" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 45631 } } peers { permanent_uuid: "791cb07119a6494a87457d8b7a0b9b84" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 37763 } } peers { permanent_uuid: "808e8ebe19f94e9485fd133fed9a68f3" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 34667 } }
I20260812 03:12:21.918129 16022 raft_consensus.cc:348] T f2abbf7df7ed47c1a6938d466999bef6 P f7feab4f02f74c138c5b704ed5c89297 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7feab4f02f74c138c5b704ed5c89297" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 45631 } } peers { permanent_uuid: "791cb07119a6494a87457d8b7a0b9b84" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 37763 } } peers { permanent_uuid: "808e8ebe19f94e9485fd133fed9a68f3" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 34667 } }
I20260812 03:12:21.918370 16021 raft_consensus.cc:374] T f2abbf7df7ed47c1a6938d466999bef6 P 791cb07119a6494a87457d8b7a0b9b84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:21.918394 16022 raft_consensus.cc:374] T f2abbf7df7ed47c1a6938d466999bef6 P f7feab4f02f74c138c5b704ed5c89297 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:21.918391 16023 raft_consensus.cc:374] T f2abbf7df7ed47c1a6938d466999bef6 P 808e8ebe19f94e9485fd133fed9a68f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:21.918444 16022 raft_consensus.cc:729] T f2abbf7df7ed47c1a6938d466999bef6 P f7feab4f02f74c138c5b704ed5c89297 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7feab4f02f74c138c5b704ed5c89297, State: Initialized, Role: FOLLOWER
I20260812 03:12:21.918442 16021 raft_consensus.cc:729] T f2abbf7df7ed47c1a6938d466999bef6 P 791cb07119a6494a87457d8b7a0b9b84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 791cb07119a6494a87457d8b7a0b9b84, State: Initialized, Role: FOLLOWER
I20260812 03:12:21.918455 16023 raft_consensus.cc:729] T f2abbf7df7ed47c1a6938d466999bef6 P 808e8ebe19f94e9485fd133fed9a68f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 808e8ebe19f94e9485fd133fed9a68f3, State: Initialized, Role: FOLLOWER
I20260812 03:12:21.918947 16023 consensus_queue.cc:260] T f2abbf7df7ed47c1a6938d466999bef6 P 808e8ebe19f94e9485fd133fed9a68f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7feab4f02f74c138c5b704ed5c89297" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 45631 } } peers { permanent_uuid: "791cb07119a6494a87457d8b7a0b9b84" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 37763 } } peers { permanent_uuid: "808e8ebe19f94e9485fd133fed9a68f3" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 34667 } }
I20260812 03:12:21.918948 16021 consensus_queue.cc:260] T f2abbf7df7ed47c1a6938d466999bef6 P 791cb07119a6494a87457d8b7a0b9b84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7feab4f02f74c138c5b704ed5c89297" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 45631 } } peers { permanent_uuid: "791cb07119a6494a87457d8b7a0b9b84" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 37763 } } peers { permanent_uuid: "808e8ebe19f94e9485fd133fed9a68f3" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 34667 } }
I20260812 03:12:21.918948 16022 consensus_queue.cc:260] T f2abbf7df7ed47c1a6938d466999bef6 P f7feab4f02f74c138c5b704ed5c89297 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7feab4f02f74c138c5b704ed5c89297" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 45631 } } peers { permanent_uuid: "791cb07119a6494a87457d8b7a0b9b84" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 37763 } } peers { permanent_uuid: "808e8ebe19f94e9485fd133fed9a68f3" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 34667 } }
I20260812 03:12:21.919790 15710 heartbeater.cc:503] Master 127.26.250.190:40095 was elected leader, sending a full tablet report...
I20260812 03:12:21.920156 16022 ts_tablet_manager.cc:1428] T f2abbf7df7ed47c1a6938d466999bef6 P f7feab4f02f74c138c5b704ed5c89297: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 03:12:21.920171 16023 ts_tablet_manager.cc:1428] T f2abbf7df7ed47c1a6938d466999bef6 P 808e8ebe19f94e9485fd133fed9a68f3: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20260812 03:12:21.920495 15861 heartbeater.cc:503] Master 127.26.250.190:40095 was elected leader, sending a full tablet report...
I20260812 03:12:21.920506 16004 heartbeater.cc:503] Master 127.26.250.190:40095 was elected leader, sending a full tablet report...
I20260812 03:12:21.920722 16021 ts_tablet_manager.cc:1428] T f2abbf7df7ed47c1a6938d466999bef6 P 791cb07119a6494a87457d8b7a0b9b84: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260812 03:12:21.958071 16028 raft_consensus.cc:482] T f2abbf7df7ed47c1a6938d466999bef6 P f7feab4f02f74c138c5b704ed5c89297 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:12:21.958410 16028 raft_consensus.cc:504] T f2abbf7df7ed47c1a6938d466999bef6 P f7feab4f02f74c138c5b704ed5c89297 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7feab4f02f74c138c5b704ed5c89297" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 45631 } } peers { permanent_uuid: "791cb07119a6494a87457d8b7a0b9b84" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 37763 } } peers { permanent_uuid: "808e8ebe19f94e9485fd133fed9a68f3" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 34667 } }
I20260812 03:12:21.959683 16028 leader_election.cc:288] T f2abbf7df7ed47c1a6938d466999bef6 P f7feab4f02f74c138c5b704ed5c89297 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 791cb07119a6494a87457d8b7a0b9b84 (127.26.250.130:37763), 808e8ebe19f94e9485fd133fed9a68f3 (127.26.250.131:34667)
I20260812 03:12:21.964154 16019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:21.960187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:51723 (local address 127.26.250.130:37763)
0812 03:12:21.960332 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:21.960337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:21.960730 (+ 393us) server_negotiation.cc:407] Connection header received
0812 03:12:21.960840 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:21.960843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:21.960902 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:21.961004 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:21.961361 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.961873 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.963126 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.963553 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.963696 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:21.963753 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:21.963939 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 03:12:21.964018 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:12:21.964177 16031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:21.960098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:37763 (local address 127.26.250.129:51723)
0812 03:12:21.960604 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:21.960618 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:21.960742 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:21.961042 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:21.961047 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:21.961066 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:21.961217 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:21.961223 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.961991 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.961997 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:21.962982 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:21.962992 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.963696 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.963700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:21.963810 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:21.963829 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 03:12:21.963921 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 03:12:21.963994 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"mutex_wait_us":1,"thread_start_us":178,"threads_started":1}
I20260812 03:12:21.964640 16020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:21.960187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:37035 (local address 127.26.250.131:34667)
0812 03:12:21.960420 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:21.960425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:21.961006 (+ 581us) server_negotiation.cc:407] Connection header received
0812 03:12:21.961121 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:21.961129 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:21.961184 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:21.961299 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:21.961661 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.962201 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.963465 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.963931 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.964090 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:21.964147 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:21.964406 (+ 259us) server_negotiation.cc:299] Negotiation successful
0812 03:12:21.964516 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 03:12:21.965054 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:21.960098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:34667 (local address 127.26.250.129:37035)
0812 03:12:21.960898 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:21.960908 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:21.961018 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:21.961350 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:21.961353 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:21.961371 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:21.961515 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:21.961521 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.962326 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.962332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:21.963319 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:21.963330 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.964074 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.964078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:21.964203 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:21.964252 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 03:12:21.964813 (+ 561us) client_negotiation.cc:240] Negotiation successful
0812 03:12:21.964922 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"mutex_wait_us":84,"thread_start_us":137,"threads_started":1}
I20260812 03:12:21.965070 15808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2abbf7df7ed47c1a6938d466999bef6" candidate_uuid: "f7feab4f02f74c138c5b704ed5c89297" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "791cb07119a6494a87457d8b7a0b9b84" is_pre_election: true
I20260812 03:12:21.965569 15808 raft_consensus.cc:2454] T f2abbf7df7ed47c1a6938d466999bef6 P 791cb07119a6494a87457d8b7a0b9b84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7feab4f02f74c138c5b704ed5c89297 in term 0.
I20260812 03:12:21.965842 15953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2abbf7df7ed47c1a6938d466999bef6" candidate_uuid: "f7feab4f02f74c138c5b704ed5c89297" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "808e8ebe19f94e9485fd133fed9a68f3" is_pre_election: true
I20260812 03:12:21.966231 15591 leader_election.cc:302] T f2abbf7df7ed47c1a6938d466999bef6 P f7feab4f02f74c138c5b704ed5c89297 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 791cb07119a6494a87457d8b7a0b9b84, f7feab4f02f74c138c5b704ed5c89297; no voters:
I20260812 03:12:21.966421 15953 raft_consensus.cc:2454] T f2abbf7df7ed47c1a6938d466999bef6 P 808e8ebe19f94e9485fd133fed9a68f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7feab4f02f74c138c5b704ed5c89297 in term 0.
I20260812 03:12:21.966634 16028 raft_consensus.cc:2781] T f2abbf7df7ed47c1a6938d466999bef6 P f7feab4f02f74c138c5b704ed5c89297 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:12:21.966753 16028 raft_consensus.cc:482] T f2abbf7df7ed47c1a6938d466999bef6 P f7feab4f02f74c138c5b704ed5c89297 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:12:21.966806 16028 raft_consensus.cc:3037] T f2abbf7df7ed47c1a6938d466999bef6 P f7feab4f02f74c138c5b704ed5c89297 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:21.967975 16028 raft_consensus.cc:504] T f2abbf7df7ed47c1a6938d466999bef6 P f7feab4f02f74c138c5b704ed5c89297 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7feab4f02f74c138c5b704ed5c89297" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 45631 } } peers { permanent_uuid: "791cb07119a6494a87457d8b7a0b9b84" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 37763 } } peers { permanent_uuid: "808e8ebe19f94e9485fd133fed9a68f3" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 34667 } }
I20260812 03:12:21.968401 16028 leader_election.cc:288] T f2abbf7df7ed47c1a6938d466999bef6 P f7feab4f02f74c138c5b704ed5c89297 [CANDIDATE]: Term 1 election: Requested vote from peers 791cb07119a6494a87457d8b7a0b9b84 (127.26.250.130:37763), 808e8ebe19f94e9485fd133fed9a68f3 (127.26.250.131:34667)
I20260812 03:12:21.968889 15953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2abbf7df7ed47c1a6938d466999bef6" candidate_uuid: "f7feab4f02f74c138c5b704ed5c89297" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "808e8ebe19f94e9485fd133fed9a68f3"
I20260812 03:12:21.968848 15808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2abbf7df7ed47c1a6938d466999bef6" candidate_uuid: "f7feab4f02f74c138c5b704ed5c89297" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "791cb07119a6494a87457d8b7a0b9b84"
I20260812 03:12:21.969033 15953 raft_consensus.cc:3037] T f2abbf7df7ed47c1a6938d466999bef6 P 808e8ebe19f94e9485fd133fed9a68f3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:21.969058 15808 raft_consensus.cc:3037] T f2abbf7df7ed47c1a6938d466999bef6 P 791cb07119a6494a87457d8b7a0b9b84 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:21.970223 15953 raft_consensus.cc:2454] T f2abbf7df7ed47c1a6938d466999bef6 P 808e8ebe19f94e9485fd133fed9a68f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7feab4f02f74c138c5b704ed5c89297 in term 1.
I20260812 03:12:21.970561 15808 raft_consensus.cc:2454] T f2abbf7df7ed47c1a6938d466999bef6 P 791cb07119a6494a87457d8b7a0b9b84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7feab4f02f74c138c5b704ed5c89297 in term 1.
I20260812 03:12:21.970686 15592 leader_election.cc:302] T f2abbf7df7ed47c1a6938d466999bef6 P f7feab4f02f74c138c5b704ed5c89297 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 808e8ebe19f94e9485fd133fed9a68f3, f7feab4f02f74c138c5b704ed5c89297; no voters:
I20260812 03:12:21.971042 16028 raft_consensus.cc:2781] T f2abbf7df7ed47c1a6938d466999bef6 P f7feab4f02f74c138c5b704ed5c89297 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:21.971333 16028 raft_consensus.cc:686] T f2abbf7df7ed47c1a6938d466999bef6 P f7feab4f02f74c138c5b704ed5c89297 [term 1 LEADER]: Becoming Leader. State: Replica: f7feab4f02f74c138c5b704ed5c89297, State: Running, Role: LEADER
I20260812 03:12:21.971698 16028 consensus_queue.cc:237] T f2abbf7df7ed47c1a6938d466999bef6 P f7feab4f02f74c138c5b704ed5c89297 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7feab4f02f74c138c5b704ed5c89297" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 45631 } } peers { permanent_uuid: "791cb07119a6494a87457d8b7a0b9b84" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 37763 } } peers { permanent_uuid: "808e8ebe19f94e9485fd133fed9a68f3" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 34667 } }
I20260812 03:12:21.975521 15507 catalog_manager.cc:5495] T f2abbf7df7ed47c1a6938d466999bef6 P f7feab4f02f74c138c5b704ed5c89297 reported cstate change: term changed from 0 to 1, leader changed from <none> to f7feab4f02f74c138c5b704ed5c89297 (127.26.250.129). New cstate: current_term: 1 leader_uuid: "f7feab4f02f74c138c5b704ed5c89297" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7feab4f02f74c138c5b704ed5c89297" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 45631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "791cb07119a6494a87457d8b7a0b9b84" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 37763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "808e8ebe19f94e9485fd133fed9a68f3" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 34667 } health_report { overall_health: UNKNOWN } } }
I20260812 03:12:21.995466 16016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:21.991987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50514 (local address 127.26.250.129:45631)
0812 03:12:21.992127 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:21.992130 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:21.992143 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:12:21.992223 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:21.992225 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:21.992290 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:21.992382 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:12:21.992643 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.993152 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.993982 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:21.994201 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:21.994293 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:21.994607 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:12:21.995058 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:12:21.995159 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:21.995218 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 03:12:21.995274 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:12:22.002059 15953 raft_consensus.cc:1261] T f2abbf7df7ed47c1a6938d466999bef6 P 808e8ebe19f94e9485fd133fed9a68f3 [term 1 FOLLOWER]: Refusing update from remote peer f7feab4f02f74c138c5b704ed5c89297: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:12:22.002059 15808 raft_consensus.cc:1261] T f2abbf7df7ed47c1a6938d466999bef6 P 791cb07119a6494a87457d8b7a0b9b84 [term 1 FOLLOWER]: Refusing update from remote peer f7feab4f02f74c138c5b704ed5c89297: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:12:22.002866 16028 consensus_queue.cc:1035] T f2abbf7df7ed47c1a6938d466999bef6 P f7feab4f02f74c138c5b704ed5c89297 [LEADER]: Connected to new peer: Peer: permanent_uuid: "808e8ebe19f94e9485fd133fed9a68f3" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 34667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:12:22.003157 16033 consensus_queue.cc:1035] T f2abbf7df7ed47c1a6938d466999bef6 P f7feab4f02f74c138c5b704ed5c89297 [LEADER]: Connected to new peer: Peer: permanent_uuid: "791cb07119a6494a87457d8b7a0b9b84" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 37763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:12:22.011763 16037 mvcc.cc:205] Tried to move back new op lower bound from 7317521784832782336 to 7317521784719245312. Current Snapshot: MvccSnapshot[applied={T|T < 7317521784832782336}]
I20260812 03:12:22.022397 15507 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:12:22.022590 15507 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:12:22.025425 15507 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69cf972ff98647c4b44aba4028cb6a3f: Sending DeleteTablet for 3 replicas of tablet f2abbf7df7ed47c1a6938d466999bef6
I20260812 03:12:22.026324 15930 tablet_service.cc:1514] Processing DeleteTablet for tablet f2abbf7df7ed47c1a6938d466999bef6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:12:22 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:50654
I20260812 03:12:22.026407 15639 tablet_service.cc:1514] Processing DeleteTablet for tablet f2abbf7df7ed47c1a6938d466999bef6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:12:22 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:50506
I20260812 03:12:22.027113 16046 tablet_replica.cc:330] T f2abbf7df7ed47c1a6938d466999bef6 P 808e8ebe19f94e9485fd133fed9a68f3: stopping tablet replica
I20260812 03:12:22.027225 15783 tablet_service.cc:1514] Processing DeleteTablet for tablet f2abbf7df7ed47c1a6938d466999bef6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:12:22 UTC) from {username='kudu', principal='kudu/127.26.250.190@KRBTEST.COM'} at 127.0.0.1:57874
I20260812 03:12:22.027474 16046 raft_consensus.cc:2229] T f2abbf7df7ed47c1a6938d466999bef6 P 808e8ebe19f94e9485fd133fed9a68f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:12:22.027842 16046 raft_consensus.cc:2258] T f2abbf7df7ed47c1a6938d466999bef6 P 808e8ebe19f94e9485fd133fed9a68f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:12:22.028435 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 15573
I20260812 03:12:22.032050 16046 ts_tablet_manager.cc:1905] T f2abbf7df7ed47c1a6938d466999bef6 P 808e8ebe19f94e9485fd133fed9a68f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:12:22.034999 16046 ts_tablet_manager.cc:1918] T f2abbf7df7ed47c1a6938d466999bef6 P 808e8ebe19f94e9485fd133fed9a68f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:12:22.035130 16046 log.cc:1196] T f2abbf7df7ed47c1a6938d466999bef6 P 808e8ebe19f94e9485fd133fed9a68f3: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/f2abbf7df7ed47c1a6938d466999bef6
I20260812 03:12:22.035487 16046 ts_tablet_manager.cc:1939] T f2abbf7df7ed47c1a6938d466999bef6 P 808e8ebe19f94e9485fd133fed9a68f3: Deleting consensus metadata
I20260812 03:12:22.035475 16048 tablet_replica.cc:330] T f2abbf7df7ed47c1a6938d466999bef6 P 791cb07119a6494a87457d8b7a0b9b84: stopping tablet replica
I20260812 03:12:22.035914 16048 raft_consensus.cc:2229] T f2abbf7df7ed47c1a6938d466999bef6 P 791cb07119a6494a87457d8b7a0b9b84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:12:22.036324 16048 raft_consensus.cc:2258] T f2abbf7df7ed47c1a6938d466999bef6 P 791cb07119a6494a87457d8b7a0b9b84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:12:22.037393 15491 catalog_manager.cc:4841] TS 808e8ebe19f94e9485fd133fed9a68f3 (127.26.250.131:34667): tablet f2abbf7df7ed47c1a6938d466999bef6 (table test-table [id=950b500dc76f41178fc1647eb4a8b645]) successfully deleted
I20260812 03:12:22.038421 16048 ts_tablet_manager.cc:1905] T f2abbf7df7ed47c1a6938d466999bef6 P 791cb07119a6494a87457d8b7a0b9b84: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 03:12:22.038812 15490 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.250.129:45631 (error 108)
I20260812 03:12:22.039844 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 15715
I20260812 03:12:22.039954 16015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:22.039627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:45631 (local address 127.0.0.1:50524)
0812 03:12:22.039780 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:22.039857 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.129:45631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 03:12:22.047102 15490 catalog_manager.cc:4584] TS f7feab4f02f74c138c5b704ed5c89297 (127.26.250.129:45631): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f2abbf7df7ed47c1a6938d466999bef6: Network error: Client connection negotiation failed: client connection to 127.26.250.129:45631: connect: Connection refused (error 111)
I20260812 03:12:22.054309 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 15866
I20260812 03:12:22.057363 16015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:22.056941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:37763 (local address 127.0.0.1:57878)
0812 03:12:22.057137 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:22.057223 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.130:37763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 03:12:22.063270 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 15471
2026-08-12T03:12:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2537 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T03:12:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:12:22Z Disabled control of system clock
I20260812 03:12:22.099211 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:45153
--webserver_interface=127.26.250.190
--webserver_port=0
--builtin_ntp_servers=127.26.250.148:36833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:45153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:12:22.224119 16053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:22.224439 16053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:22.224476 16053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:22.228822 16053 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:22.228909 16053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:22.228924 16053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:22.228942 16053 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:22.228957 16053 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:22.234426 16053 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:36833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:45153
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:45153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:22.235757 16053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:22.237303 16053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:22.246639 16060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:22.246639 16059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:22.246665 16063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:22.246932 16053 server_base.cc:1034] running on GCE node
I20260812 03:12:22.247807 16053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:22.249223 16053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:22.250429 16053 hybrid_clock.cc:648] HybridClock initialized: now 1786504342250414 us; error 34 us; skew 500 ppm
I20260812 03:12:22.252571 16053 webserver.cc:460] Webserver started at http://127.26.250.190:40071/ using document root <none> and password file <none>
I20260812 03:12:22.253291 16053 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:22.253396 16053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:22.253643 16053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:22.256115 16053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "f9f967c9fc214d779376874b730cb6d3"
format_stamp: "Formatted at 2026-08-12 03:12:22 on dist-test-slave-zpfg"
server_key: "02e01f1eba617e6d28dcfcd94c3af4c3"
server_key_iv: "cb07d9cad3aec2457940f9df306cd0cd"
server_key_version: "encryptionkey@0"
I20260812 03:12:22.256793 16053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "f9f967c9fc214d779376874b730cb6d3"
format_stamp: "Formatted at 2026-08-12 03:12:22 on dist-test-slave-zpfg"
server_key: "02e01f1eba617e6d28dcfcd94c3af4c3"
server_key_iv: "cb07d9cad3aec2457940f9df306cd0cd"
server_key_version: "encryptionkey@0"
I20260812 03:12:22.261147 16053 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:12:22.264084 16069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:22.265733 16053 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:12:22.265956 16053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "f9f967c9fc214d779376874b730cb6d3"
format_stamp: "Formatted at 2026-08-12 03:12:22 on dist-test-slave-zpfg"
server_key: "02e01f1eba617e6d28dcfcd94c3af4c3"
server_key_iv: "cb07d9cad3aec2457940f9df306cd0cd"
server_key_version: "encryptionkey@0"
I20260812 03:12:22.266098 16053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:22.280054 16053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:22.280864 16053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:22.281082 16053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:22.289021 16053 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:45153
I20260812 03:12:22.289093 16129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:45153 every 8 connection(s)
I20260812 03:12:22.290108 16053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:12:22.294230 16130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:22.296855 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 16053
I20260812 03:12:22.297012 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:12:22.297370 27626 external_mini_cluster.cc:1428] Setting key 28ca3534904b544702f6d6f36610dee9
I20260812 03:12:22.301753 16130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9f967c9fc214d779376874b730cb6d3: Bootstrap starting.
I20260812 03:12:22.304358 16130 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9f967c9fc214d779376874b730cb6d3: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:22.305270 16130 log.cc:824] T 00000000000000000000000000000000 P f9f967c9fc214d779376874b730cb6d3: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:22.307983 16130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9f967c9fc214d779376874b730cb6d3: No bootstrap required, opened a new log
I20260812 03:12:22.308104 16133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:22.299116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56244 (local address 127.26.250.190:45153)
0812 03:12:22.299889 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:22.299905 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:22.299945 (+ 40us) server_negotiation.cc:407] Connection header received
0812 03:12:22.300751 (+ 806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:22.300773 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:22.301218 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:22.301589 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:22.302051 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:22.303131 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:22.304062 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:22.304620 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:22.304799 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:22.305819 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:22.305825 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:22.305842 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:22.306385 (+ 543us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:22.306466 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:22.306495 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:22.306609 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:22.306755 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 03:12:22.307079 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":182,"threads_started":1}
I20260812 03:12:22.312047 16130 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9f967c9fc214d779376874b730cb6d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f967c9fc214d779376874b730cb6d3" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45153 } }
I20260812 03:12:22.312302 16130 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9f967c9fc214d779376874b730cb6d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:22.312371 16130 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9f967c9fc214d779376874b730cb6d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9f967c9fc214d779376874b730cb6d3, State: Initialized, Role: FOLLOWER
I20260812 03:12:22.312813 16130 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9f967c9fc214d779376874b730cb6d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f967c9fc214d779376874b730cb6d3" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45153 } }
I20260812 03:12:22.312997 16130 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9f967c9fc214d779376874b730cb6d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:12:22.313094 16130 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9f967c9fc214d779376874b730cb6d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:12:22.313225 16130 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9f967c9fc214d779376874b730cb6d3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:22.314301 16130 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9f967c9fc214d779376874b730cb6d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f967c9fc214d779376874b730cb6d3" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45153 } }
I20260812 03:12:22.314743 16130 leader_election.cc:302] T 00000000000000000000000000000000 P f9f967c9fc214d779376874b730cb6d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9f967c9fc214d779376874b730cb6d3; no voters:
I20260812 03:12:22.315117 16130 leader_election.cc:288] T 00000000000000000000000000000000 P f9f967c9fc214d779376874b730cb6d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:12:22.315259 16136 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9f967c9fc214d779376874b730cb6d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:22.315513 16136 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9f967c9fc214d779376874b730cb6d3 [term 1 LEADER]: Becoming Leader. State: Replica: f9f967c9fc214d779376874b730cb6d3, State: Running, Role: LEADER
I20260812 03:12:22.315848 16136 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9f967c9fc214d779376874b730cb6d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f967c9fc214d779376874b730cb6d3" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45153 } }
I20260812 03:12:22.316526 16130 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9f967c9fc214d779376874b730cb6d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:22.317698 16137 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9f967c9fc214d779376874b730cb6d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9f967c9fc214d779376874b730cb6d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f967c9fc214d779376874b730cb6d3" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45153 } } }
I20260812 03:12:22.317847 16137 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9f967c9fc214d779376874b730cb6d3 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:22.318261 16140 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9f967c9fc214d779376874b730cb6d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9f967c9fc214d779376874b730cb6d3. Latest consensus state: current_term: 1 leader_uuid: "f9f967c9fc214d779376874b730cb6d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f967c9fc214d779376874b730cb6d3" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45153 } } }
I20260812 03:12:22.318459 16140 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9f967c9fc214d779376874b730cb6d3 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:22.319267 16147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:12:22.322309 16147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:12:22.328899 16147 catalog_manager.cc:1347] Generated new cluster ID: 4439340f15814fc9a4a7390c18cca732
I20260812 03:12:22.328991 16147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:12:22.342609 16147 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:12:22.343981 16147 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:12:22.358126 16147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9f967c9fc214d779376874b730cb6d3: Generated new TSK 0
I20260812 03:12:22.359490 16147 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:12:22.374604 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.129:0
--local_ip_for_outbound_sockets=127.26.250.129
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:45153
--builtin_ntp_servers=127.26.250.148:36833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 03:12:22.497900 16157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:22.498227 16157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:22.498271 16157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:22.502771 16157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:22.502851 16157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:22.502912 16157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.129
I20260812 03:12:22.507961 16157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:36833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.26.250.129
--webserver_port=0
--tserver_master_addrs=127.26.250.190:45153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.129
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:22.509188 16157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:22.510639 16157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:22.521687 16163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:22.521708 16165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:22.521770 16157 server_base.cc:1034] running on GCE node
W20260812 03:12:22.521693 16162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:22.522564 16157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:22.523305 16157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:22.524487 16157 hybrid_clock.cc:648] HybridClock initialized: now 1786504342524454 us; error 44 us; skew 500 ppm
I20260812 03:12:22.526613 16157 webserver.cc:460] Webserver started at http://127.26.250.129:38657/ using document root <none> and password file <none>
I20260812 03:12:22.527237 16157 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:22.527338 16157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:22.527587 16157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:22.529764 16157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-0/data/instance:
uuid: "207b2c95ee544e16bafb3691d331f58e"
format_stamp: "Formatted at 2026-08-12 03:12:22 on dist-test-slave-zpfg"
server_key: "2727c058caa5347ddf33b1a6bcb05894"
server_key_iv: "bcdca53122208cc34497199a8d908adb"
server_key_version: "encryptionkey@0"
I20260812 03:12:22.530429 16157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance:
uuid: "207b2c95ee544e16bafb3691d331f58e"
format_stamp: "Formatted at 2026-08-12 03:12:22 on dist-test-slave-zpfg"
server_key: "2727c058caa5347ddf33b1a6bcb05894"
server_key_iv: "bcdca53122208cc34497199a8d908adb"
server_key_version: "encryptionkey@0"
I20260812 03:12:22.534744 16157 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:12:22.537600 16171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:22.539085 16157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:12:22.539243 16157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-0/wal
uuid: "207b2c95ee544e16bafb3691d331f58e"
format_stamp: "Formatted at 2026-08-12 03:12:22 on dist-test-slave-zpfg"
server_key: "2727c058caa5347ddf33b1a6bcb05894"
server_key_iv: "bcdca53122208cc34497199a8d908adb"
server_key_version: "encryptionkey@0"
I20260812 03:12:22.539381 16157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:22.555753 16157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:22.556737 16157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:22.556998 16157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:22.557699 16157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:12:22.558907 16157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:12:22.558988 16157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:22.559068 16157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:12:22.559116 16157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:22.570258 16289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.129:36225 every 8 connection(s)
I20260812 03:12:22.570267 16157 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.129:36225
I20260812 03:12:22.571383 16157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-0/data/info.pb
I20260812 03:12:22.572793 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 16157
I20260812 03:12:22.572955 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-0/wal/instance
I20260812 03:12:22.573380 27626 external_mini_cluster.cc:1428] Setting key 0d0dea72e08f1e57f5199b8c969a72be
I20260812 03:12:22.575744 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.130:0
--local_ip_for_outbound_sockets=127.26.250.130
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:45153
--builtin_ntp_servers=127.26.250.148:36833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 03:12:22.583374 16133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:22.573670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.129:54139 (local address 127.26.250.190:45153)
0812 03:12:22.573835 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:22.573838 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:22.575397 (+ 1559us) server_negotiation.cc:407] Connection header received
0812 03:12:22.577153 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:22.577160 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:22.577241 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:22.577347 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:22.579109 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:22.579692 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:22.581150 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:22.581391 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:22.581515 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:22.582826 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:22.582830 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:22.582832 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:22.582855 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:22.582876 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:22.582885 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:22.582965 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:22.583142 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 03:12:22.583217 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 03:12:22.584488 16292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:22.574213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:45153 (local address 127.26.250.129:54139)
0812 03:12:22.575213 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:22.575275 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:22.576888 (+ 1613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:22.577651 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:22.577666 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:22.577893 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:22.578871 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:22.578900 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:22.579927 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:22.579931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:22.580985 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:22.580997 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:22.581580 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:22.581584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:22.581772 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:22.582521 (+ 749us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:12:22.582523 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:22.582571 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:12:22.582648 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:22.582969 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:22.582982 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:22.582988 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 03:12:22.583130 (+ 142us) client_negotiation.cc:240] Negotiation successful
0812 03:12:22.583604 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":155,"threads_started":1}
I20260812 03:12:22.586019 16290 heartbeater.cc:347] Connected to a master server at 127.26.250.190:45153
I20260812 03:12:22.586407 16290 heartbeater.cc:464] Registering TS with master...
I20260812 03:12:22.587212 16290 heartbeater.cc:511] Master 127.26.250.190:45153 requested a full tablet report, sending...
I20260812 03:12:22.589077 16087 ts_manager.cc:194] Registered new tserver with Master: 207b2c95ee544e16bafb3691d331f58e (127.26.250.129:36225)
I20260812 03:12:22.590458 16087 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.250.129:54139
W20260812 03:12:22.710551 16294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:22.710876 16294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:22.710922 16294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:22.715873 16294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:22.715946 16294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:22.716040 16294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.130
I20260812 03:12:22.721890 16294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:36833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.26.250.130
--webserver_port=0
--tserver_master_addrs=127.26.250.190:45153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.130
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:22.723656 16294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:22.725301 16294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:22.736198 16300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:22.736191 16299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:22.736194 16302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:22.736303 16294 server_base.cc:1034] running on GCE node
I20260812 03:12:22.736938 16294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:22.737603 16294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:22.738875 16294 hybrid_clock.cc:648] HybridClock initialized: now 1786504342738854 us; error 35 us; skew 500 ppm
I20260812 03:12:22.741078 16294 webserver.cc:460] Webserver started at http://127.26.250.130:43659/ using document root <none> and password file <none>
I20260812 03:12:22.741832 16294 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:22.741941 16294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:22.742199 16294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:22.744676 16294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-1/data/instance:
uuid: "ff15f90157e14440ad85424e809273b9"
format_stamp: "Formatted at 2026-08-12 03:12:22 on dist-test-slave-zpfg"
server_key: "7bb89ce2fff2f1e1ced32abf0cb61762"
server_key_iv: "8d41910c2236b10374aad1804f46d832"
server_key_version: "encryptionkey@0"
I20260812 03:12:22.745321 16294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance:
uuid: "ff15f90157e14440ad85424e809273b9"
format_stamp: "Formatted at 2026-08-12 03:12:22 on dist-test-slave-zpfg"
server_key: "7bb89ce2fff2f1e1ced32abf0cb61762"
server_key_iv: "8d41910c2236b10374aad1804f46d832"
server_key_version: "encryptionkey@0"
I20260812 03:12:22.749447 16294 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:12:22.752242 16308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:22.753597 16294 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:12:22.753746 16294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-1/wal
uuid: "ff15f90157e14440ad85424e809273b9"
format_stamp: "Formatted at 2026-08-12 03:12:22 on dist-test-slave-zpfg"
server_key: "7bb89ce2fff2f1e1ced32abf0cb61762"
server_key_iv: "8d41910c2236b10374aad1804f46d832"
server_key_version: "encryptionkey@0"
I20260812 03:12:22.753873 16294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:22.786006 16294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:22.786885 16294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:22.787103 16294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:22.787781 16294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:12:22.788897 16294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:12:22.788952 16294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:22.789024 16294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:12:22.789064 16294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:22.798727 16294 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.130:41139
I20260812 03:12:22.798748 16431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.130:41139 every 8 connection(s)
I20260812 03:12:22.799755 16294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-1/data/info.pb
I20260812 03:12:22.805567 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 16294
I20260812 03:12:22.805706 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-1/wal/instance
I20260812 03:12:22.806020 27626 external_mini_cluster.cc:1428] Setting key 5192b6c8d5d8dbcbe4f90095269c3d48
I20260812 03:12:22.808025 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.26.250.131:0
--local_ip_for_outbound_sockets=127.26.250.131
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:45153
--builtin_ntp_servers=127.26.250.148:36833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 03:12:22.810135 16133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:22.801669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.130:41349 (local address 127.26.250.190:45153)
0812 03:12:22.801841 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:22.801844 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:22.802987 (+ 1143us) server_negotiation.cc:407] Connection header received
0812 03:12:22.804475 (+ 1488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:22.804478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:22.804538 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:22.804652 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:22.805612 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:22.806332 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:22.807889 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:22.808183 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:22.808289 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:22.809490 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:22.809494 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:22.809497 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:22.809511 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:22.809536 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:22.809548 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:22.809629 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:22.809869 (+ 240us) server_negotiation.cc:299] Negotiation successful
0812 03:12:22.809942 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 03:12:22.811036 16434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:22.802100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:45153 (local address 127.26.250.130:41349)
0812 03:12:22.802832 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:22.802876 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:22.804236 (+ 1360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:22.804835 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:22.804843 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:22.804932 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:22.805462 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:22.805475 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:22.806491 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:22.806499 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:22.807761 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:22.807771 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:22.808344 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:22.808348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:22.808516 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:22.809206 (+ 690us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:12:22.809208 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:22.809251 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:12:22.809327 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:22.809668 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:22.809682 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:22.809691 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 03:12:22.809855 (+ 164us) client_negotiation.cc:240] Negotiation successful
0812 03:12:22.810099 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":107,"threads_started":1}
I20260812 03:12:22.812301 16432 heartbeater.cc:347] Connected to a master server at 127.26.250.190:45153
I20260812 03:12:22.812588 16432 heartbeater.cc:464] Registering TS with master...
I20260812 03:12:22.813179 16432 heartbeater.cc:511] Master 127.26.250.190:45153 requested a full tablet report, sending...
I20260812 03:12:22.814309 16087 ts_manager.cc:194] Registered new tserver with Master: ff15f90157e14440ad85424e809273b9 (127.26.250.130:41139)
I20260812 03:12:22.814975 16087 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.250.130:41349
W20260812 03:12:22.930243 16435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:22.930627 16435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:22.930706 16435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:22.935114 16435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:22.935209 16435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:22.935312 16435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.26.250.131
I20260812 03:12:22.940537 16435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:36833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.26.250.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.26.250.131
--webserver_port=0
--tserver_master_addrs=127.26.250.190:45153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.26.250.131
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:22.941847 16435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:22.943392 16435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:22.953603 16443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:22.953603 16442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:22.953655 16435 server_base.cc:1034] running on GCE node
W20260812 03:12:22.953603 16445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:22.954308 16435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:22.955032 16435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:22.956246 16435 hybrid_clock.cc:648] HybridClock initialized: now 1786504342956152 us; error 113 us; skew 500 ppm
I20260812 03:12:22.958256 16435 webserver.cc:460] Webserver started at http://127.26.250.131:40351/ using document root <none> and password file <none>
I20260812 03:12:22.958901 16435 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:22.958998 16435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:22.959240 16435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:22.961427 16435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-2/data/instance:
uuid: "f4f919c590a94cf7b6f699e829e6d86b"
format_stamp: "Formatted at 2026-08-12 03:12:22 on dist-test-slave-zpfg"
server_key: "f766b280a7deeb85b0a2cccb2097848c"
server_key_iv: "88f09b7a88002c21c070c155c3f8ea1d"
server_key_version: "encryptionkey@0"
I20260812 03:12:22.962056 16435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance:
uuid: "f4f919c590a94cf7b6f699e829e6d86b"
format_stamp: "Formatted at 2026-08-12 03:12:22 on dist-test-slave-zpfg"
server_key: "f766b280a7deeb85b0a2cccb2097848c"
server_key_iv: "88f09b7a88002c21c070c155c3f8ea1d"
server_key_version: "encryptionkey@0"
I20260812 03:12:22.966091 16435 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 03:12:22.968763 16451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:22.970168 16435 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:12:22.970319 16435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-2/wal
uuid: "f4f919c590a94cf7b6f699e829e6d86b"
format_stamp: "Formatted at 2026-08-12 03:12:22 on dist-test-slave-zpfg"
server_key: "f766b280a7deeb85b0a2cccb2097848c"
server_key_iv: "88f09b7a88002c21c070c155c3f8ea1d"
server_key_version: "encryptionkey@0"
I20260812 03:12:22.970515 16435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:22.986200 16435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:22.987025 16435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:22.987247 16435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:22.987910 16435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 03:12:22.989013 16435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 03:12:22.989084 16435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:22.989159 16435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 03:12:22.989200 16435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:22.999220 16574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.131:39891 every 8 connection(s)
I20260812 03:12:22.999310 16435 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.131:39891
I20260812 03:12:23.000308 16435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-2/data/info.pb
I20260812 03:12:23.006421 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 16435
I20260812 03:12:23.006595 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-2/wal/instance
I20260812 03:12:23.006913 27626 external_mini_cluster.cc:1428] Setting key dd4c98aa8df4c1af9a88e6e10abdaea6
I20260812 03:12:23.011508 16133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:23.002999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:34683 (local address 127.26.250.190:45153)
0812 03:12:23.003143 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:23.003146 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:23.004403 (+ 1257us) server_negotiation.cc:407] Connection header received
0812 03:12:23.006111 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:23.006115 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:23.006199 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:23.006311 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:23.007553 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.008097 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.009319 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.009562 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.009682 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:23.010946 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:23.010950 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:23.010953 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:23.010974 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:23.010994 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:23.011002 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:23.011084 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:23.011256 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 03:12:23.011355 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 03:12:23.012465 16577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:23.003393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:45153 (local address 127.26.250.131:34683)
0812 03:12:23.004198 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:23.004248 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:23.005843 (+ 1595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:23.006583 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:23.006597 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:23.006723 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:23.007382 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:23.007399 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.008241 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.008245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:23.009182 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:23.009192 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.009745 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.009752 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:23.009925 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:23.010630 (+ 705us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 03:12:23.010632 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:23.010676 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 03:12:23.010751 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:23.011099 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:23.011112 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:23.011124 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 03:12:23.011256 (+ 132us) client_negotiation.cc:240] Negotiation successful
0812 03:12:23.011579 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":159,"threads_started":1}
I20260812 03:12:23.013682 16575 heartbeater.cc:347] Connected to a master server at 127.26.250.190:45153
I20260812 03:12:23.013964 16575 heartbeater.cc:464] Registering TS with master...
I20260812 03:12:23.014595 16575 heartbeater.cc:511] Master 127.26.250.190:45153 requested a full tablet report, sending...
I20260812 03:12:23.015826 16087 ts_manager.cc:194] Registered new tserver with Master: f4f919c590a94cf7b6f699e829e6d86b (127.26.250.131:39891)
I20260812 03:12:23.016476 16087 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.26.250.131:34683
I20260812 03:12:23.021132 27626 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 03:12:23.026944 16133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:23.023326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56260 (local address 127.26.250.190:45153)
0812 03:12:23.023476 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:23.023480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:23.023643 (+ 163us) server_negotiation.cc:407] Connection header received
0812 03:12:23.023754 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:23.023756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:23.023798 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:23.023865 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:23.024127 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.024644 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.025457 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.025663 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.025739 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:23.026627 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:23.026630 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:23.026632 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:23.026644 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:23.026663 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:23.026670 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:23.026727 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:23.026777 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 03:12:23.026825 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 03:12:23.030031 16087 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56260:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 03:12:23.033203 16087 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 03:12:23.049117 16587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:23.042541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54468 (local address 127.26.250.130:41139)
0812 03:12:23.042991 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:23.043006 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:23.043398 (+ 392us) server_negotiation.cc:407] Connection header received
0812 03:12:23.043488 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:23.043496 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:23.043768 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:23.043929 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:23.044472 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.045706 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.047591 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.048406 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.048547 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:23.048760 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:23.048818 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 03:12:23.048928 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":118,"threads_started":1}
I20260812 03:12:23.049120 16586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:23.042293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:41139 (local address 127.0.0.1:54468)
0812 03:12:23.043055 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:23.043108 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:23.043241 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:23.044164 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:23.044172 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:23.044195 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:23.044357 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:23.044366 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.045832 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.045835 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:23.047452 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:23.047462 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.048534 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.048537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:23.048618 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:23.048649 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 03:12:23.048767 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 03:12:23.048943 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":339,"thread_start_us":157,"threads_started":1}
I20260812 03:12:23.049120 16588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:23.042693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.131:39891 (local address 127.0.0.1:60062)
0812 03:12:23.043452 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:23.043467 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:23.043564 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:23.044553 (+ 989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:23.044555 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:23.044573 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:23.044662 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:23.044668 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.046020 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.046027 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:23.047708 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:23.047720 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.048767 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.048771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:23.048851 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:23.048870 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 03:12:23.048931 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 03:12:23.048981 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"mutex_wait_us":113,"thread_start_us":132,"threads_started":1}
I20260812 03:12:23.049240 16590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:23.043416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60062 (local address 127.26.250.131:39891)
0812 03:12:23.044094 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:23.044105 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:23.044131 (+ 26us) server_negotiation.cc:407] Connection header received
0812 03:12:23.044215 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:23.044223 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:23.044425 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:23.044578 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:23.044787 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.045905 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.047850 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.048644 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.048764 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:23.048922 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:23.048965 (+ 43us) server_negotiation.cc:299] Negotiation successful
0812 03:12:23.049053 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":102,"threads_started":1}
I20260812 03:12:23.052016 16589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:23.043143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:36225 (local address 127.0.0.1:53520)
0812 03:12:23.044196 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:23.044209 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:23.044325 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:23.044929 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:23.044931 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:23.044945 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:23.045005 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:23.045009 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.046105 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.046109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:23.047708 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:23.047720 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.051580 (+ 3860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.051584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:23.051674 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:23.051699 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 03:12:23.051785 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 03:12:23.051864 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":930,"mutex_wait_us":186,"thread_start_us":100,"threads_started":1}
I20260812 03:12:23.052109 16591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:23.043598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53520 (local address 127.26.250.129:36225)
0812 03:12:23.043995 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:23.044002 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:23.044319 (+ 317us) server_negotiation.cc:407] Connection header received
0812 03:12:23.044436 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:23.044443 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:23.044634 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:23.044785 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:23.045124 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.045952 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.050622 (+ 4670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.051436 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.051574 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:23.051746 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:23.051828 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 03:12:23.051933 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":102,"threads_started":1}
I20260812 03:12:23.052462 16504 tablet_service.cc:1467] Processing CreateTablet for tablet 85733dae67104b429b2126a77c61d98d (DEFAULT_TABLE table=test-table [id=c0310d5c5cda4382bf9128e292f775f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:12:23.052466 16359 tablet_service.cc:1467] Processing CreateTablet for tablet 85733dae67104b429b2126a77c61d98d (DEFAULT_TABLE table=test-table [id=c0310d5c5cda4382bf9128e292f775f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:12:23.053413 16359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85733dae67104b429b2126a77c61d98d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:23.053413 16504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85733dae67104b429b2126a77c61d98d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:23.059079 16222 tablet_service.cc:1467] Processing CreateTablet for tablet 85733dae67104b429b2126a77c61d98d (DEFAULT_TABLE table=test-table [id=c0310d5c5cda4382bf9128e292f775f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 03:12:23.060505 16222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85733dae67104b429b2126a77c61d98d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:23.062322 16592 tablet_bootstrap.cc:492] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b: Bootstrap starting.
I20260812 03:12:23.063086 16593 tablet_bootstrap.cc:492] T 85733dae67104b429b2126a77c61d98d P ff15f90157e14440ad85424e809273b9: Bootstrap starting.
I20260812 03:12:23.065080 16592 tablet_bootstrap.cc:654] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:23.065279 16593 tablet_bootstrap.cc:654] T 85733dae67104b429b2126a77c61d98d P ff15f90157e14440ad85424e809273b9: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:23.066053 16592 log.cc:824] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:23.066053 16593 log.cc:824] T 85733dae67104b429b2126a77c61d98d P ff15f90157e14440ad85424e809273b9: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:23.067906 16597 tablet_bootstrap.cc:492] T 85733dae67104b429b2126a77c61d98d P 207b2c95ee544e16bafb3691d331f58e: Bootstrap starting.
I20260812 03:12:23.068729 16592 tablet_bootstrap.cc:492] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b: No bootstrap required, opened a new log
I20260812 03:12:23.068740 16593 tablet_bootstrap.cc:492] T 85733dae67104b429b2126a77c61d98d P ff15f90157e14440ad85424e809273b9: No bootstrap required, opened a new log
I20260812 03:12:23.068938 16592 ts_tablet_manager.cc:1397] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260812 03:12:23.068938 16593 ts_tablet_manager.cc:1397] T 85733dae67104b429b2126a77c61d98d P ff15f90157e14440ad85424e809273b9: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260812 03:12:23.070169 16597 tablet_bootstrap.cc:654] T 85733dae67104b429b2126a77c61d98d P 207b2c95ee544e16bafb3691d331f58e: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:23.071085 16597 log.cc:824] T 85733dae67104b429b2126a77c61d98d P 207b2c95ee544e16bafb3691d331f58e: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:23.073251 16597 tablet_bootstrap.cc:492] T 85733dae67104b429b2126a77c61d98d P 207b2c95ee544e16bafb3691d331f58e: No bootstrap required, opened a new log
I20260812 03:12:23.073540 16597 ts_tablet_manager.cc:1397] T 85733dae67104b429b2126a77c61d98d P 207b2c95ee544e16bafb3691d331f58e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260812 03:12:23.073742 16593 raft_consensus.cc:348] T 85733dae67104b429b2126a77c61d98d P ff15f90157e14440ad85424e809273b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff15f90157e14440ad85424e809273b9" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41139 } } peers { permanent_uuid: "207b2c95ee544e16bafb3691d331f58e" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 36225 } } peers { permanent_uuid: "f4f919c590a94cf7b6f699e829e6d86b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39891 } }
I20260812 03:12:23.074071 16593 raft_consensus.cc:374] T 85733dae67104b429b2126a77c61d98d P ff15f90157e14440ad85424e809273b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:23.074121 16593 raft_consensus.cc:729] T 85733dae67104b429b2126a77c61d98d P ff15f90157e14440ad85424e809273b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff15f90157e14440ad85424e809273b9, State: Initialized, Role: FOLLOWER
I20260812 03:12:23.073974 16592 raft_consensus.cc:348] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff15f90157e14440ad85424e809273b9" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41139 } } peers { permanent_uuid: "207b2c95ee544e16bafb3691d331f58e" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 36225 } } peers { permanent_uuid: "f4f919c590a94cf7b6f699e829e6d86b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39891 } }
I20260812 03:12:23.074275 16592 raft_consensus.cc:374] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:23.074331 16592 raft_consensus.cc:729] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4f919c590a94cf7b6f699e829e6d86b, State: Initialized, Role: FOLLOWER
I20260812 03:12:23.074754 16593 consensus_queue.cc:260] T 85733dae67104b429b2126a77c61d98d P ff15f90157e14440ad85424e809273b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff15f90157e14440ad85424e809273b9" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41139 } } peers { permanent_uuid: "207b2c95ee544e16bafb3691d331f58e" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 36225 } } peers { permanent_uuid: "f4f919c590a94cf7b6f699e829e6d86b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39891 } }
I20260812 03:12:23.074770 16592 consensus_queue.cc:260] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff15f90157e14440ad85424e809273b9" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41139 } } peers { permanent_uuid: "207b2c95ee544e16bafb3691d331f58e" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 36225 } } peers { permanent_uuid: "f4f919c590a94cf7b6f699e829e6d86b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39891 } }
I20260812 03:12:23.075876 16593 ts_tablet_manager.cc:1428] T 85733dae67104b429b2126a77c61d98d P ff15f90157e14440ad85424e809273b9: Time spent starting tablet: real 0.007s user 0.006s sys 0.002s
I20260812 03:12:23.075876 16592 ts_tablet_manager.cc:1428] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260812 03:12:23.076282 16432 heartbeater.cc:503] Master 127.26.250.190:45153 was elected leader, sending a full tablet report...
I20260812 03:12:23.076290 16575 heartbeater.cc:503] Master 127.26.250.190:45153 was elected leader, sending a full tablet report...
I20260812 03:12:23.077785 16597 raft_consensus.cc:348] T 85733dae67104b429b2126a77c61d98d P 207b2c95ee544e16bafb3691d331f58e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff15f90157e14440ad85424e809273b9" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41139 } } peers { permanent_uuid: "207b2c95ee544e16bafb3691d331f58e" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 36225 } } peers { permanent_uuid: "f4f919c590a94cf7b6f699e829e6d86b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39891 } }
I20260812 03:12:23.078078 16597 raft_consensus.cc:374] T 85733dae67104b429b2126a77c61d98d P 207b2c95ee544e16bafb3691d331f58e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:23.078140 16597 raft_consensus.cc:729] T 85733dae67104b429b2126a77c61d98d P 207b2c95ee544e16bafb3691d331f58e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 207b2c95ee544e16bafb3691d331f58e, State: Initialized, Role: FOLLOWER
I20260812 03:12:23.078617 16597 consensus_queue.cc:260] T 85733dae67104b429b2126a77c61d98d P 207b2c95ee544e16bafb3691d331f58e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff15f90157e14440ad85424e809273b9" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41139 } } peers { permanent_uuid: "207b2c95ee544e16bafb3691d331f58e" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 36225 } } peers { permanent_uuid: "f4f919c590a94cf7b6f699e829e6d86b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39891 } }
I20260812 03:12:23.079341 16290 heartbeater.cc:503] Master 127.26.250.190:45153 was elected leader, sending a full tablet report...
I20260812 03:12:23.079730 16597 ts_tablet_manager.cc:1428] T 85733dae67104b429b2126a77c61d98d P 207b2c95ee544e16bafb3691d331f58e: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260812 03:12:23.198935 16602 raft_consensus.cc:482] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 03:12:23.199203 16602 raft_consensus.cc:504] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff15f90157e14440ad85424e809273b9" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41139 } } peers { permanent_uuid: "207b2c95ee544e16bafb3691d331f58e" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 36225 } } peers { permanent_uuid: "f4f919c590a94cf7b6f699e829e6d86b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39891 } }
I20260812 03:12:23.200464 16602 leader_election.cc:288] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff15f90157e14440ad85424e809273b9 (127.26.250.130:41139), 207b2c95ee544e16bafb3691d331f58e (127.26.250.129:36225)
I20260812 03:12:23.204742 16577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:23.200793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.130:41139 (local address 127.26.250.131:33117)
0812 03:12:23.200972 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:23.201009 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:23.201130 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:23.201543 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:23.201547 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:23.201568 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:23.201746 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:23.201754 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.202811 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.202814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:23.203602 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:23.203611 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.204292 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.204296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:23.204403 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:23.204431 (+ 28us) client_negotiation.cc:769] Sending connection context
0812 03:12:23.204522 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 03:12:23.204597 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260812 03:12:23.204742 16587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:23.200953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:33117 (local address 127.26.250.130:41139)
0812 03:12:23.201106 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:23.201111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:23.201128 (+ 17us) server_negotiation.cc:407] Connection header received
0812 03:12:23.201298 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:23.201303 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:23.201375 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:23.201465 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:23.201904 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.202662 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.203736 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.204158 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.204299 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:23.204367 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:23.204538 (+ 171us) server_negotiation.cc:299] Negotiation successful
0812 03:12:23.204613 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 03:12:23.205262 16591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:23.201304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.26.250.131:59431 (local address 127.26.250.129:36225)
0812 03:12:23.201482 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:23.201489 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:23.201955 (+ 466us) server_negotiation.cc:407] Connection header received
0812 03:12:23.202093 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:23.202096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:23.202158 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:23.202256 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 03:12:23.202636 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.203211 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.204237 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.204638 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.204786 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:23.204843 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:23.205072 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 03:12:23.205152 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 03:12:23.205257 16604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:23.201175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:36225 (local address 127.26.250.131:59431)
0812 03:12:23.201862 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:23.201877 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:23.201978 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:23.202271 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:23.202274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:23.202290 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 03:12:23.202463 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:23.202488 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.203344 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.203346 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:23.204104 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:23.204112 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.204794 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.204798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:23.204900 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:23.204931 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 03:12:23.205049 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 03:12:23.205117 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":176,"threads_started":1}
I20260812 03:12:23.205507 16379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85733dae67104b429b2126a77c61d98d" candidate_uuid: "f4f919c590a94cf7b6f699e829e6d86b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff15f90157e14440ad85424e809273b9" is_pre_election: true
I20260812 03:12:23.205912 16379 raft_consensus.cc:2454] T 85733dae67104b429b2126a77c61d98d P ff15f90157e14440ad85424e809273b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4f919c590a94cf7b6f699e829e6d86b in term 0.
I20260812 03:12:23.205968 16242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85733dae67104b429b2126a77c61d98d" candidate_uuid: "f4f919c590a94cf7b6f699e829e6d86b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "207b2c95ee544e16bafb3691d331f58e" is_pre_election: true
I20260812 03:12:23.206297 16242 raft_consensus.cc:2454] T 85733dae67104b429b2126a77c61d98d P 207b2c95ee544e16bafb3691d331f58e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4f919c590a94cf7b6f699e829e6d86b in term 0.
I20260812 03:12:23.206523 16454 leader_election.cc:302] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f4f919c590a94cf7b6f699e829e6d86b, ff15f90157e14440ad85424e809273b9; no voters:
I20260812 03:12:23.206830 16602 raft_consensus.cc:2781] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 03:12:23.206918 16602 raft_consensus.cc:482] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 03:12:23.206950 16602 raft_consensus.cc:3037] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:23.208040 16602 raft_consensus.cc:504] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff15f90157e14440ad85424e809273b9" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41139 } } peers { permanent_uuid: "207b2c95ee544e16bafb3691d331f58e" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 36225 } } peers { permanent_uuid: "f4f919c590a94cf7b6f699e829e6d86b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39891 } }
I20260812 03:12:23.208411 16602 leader_election.cc:288] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b [CANDIDATE]: Term 1 election: Requested vote from peers ff15f90157e14440ad85424e809273b9 (127.26.250.130:41139), 207b2c95ee544e16bafb3691d331f58e (127.26.250.129:36225)
I20260812 03:12:23.208904 16242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85733dae67104b429b2126a77c61d98d" candidate_uuid: "f4f919c590a94cf7b6f699e829e6d86b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "207b2c95ee544e16bafb3691d331f58e"
I20260812 03:12:23.208890 16379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85733dae67104b429b2126a77c61d98d" candidate_uuid: "f4f919c590a94cf7b6f699e829e6d86b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff15f90157e14440ad85424e809273b9"
I20260812 03:12:23.209038 16242 raft_consensus.cc:3037] T 85733dae67104b429b2126a77c61d98d P 207b2c95ee544e16bafb3691d331f58e [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:23.209038 16379 raft_consensus.cc:3037] T 85733dae67104b429b2126a77c61d98d P ff15f90157e14440ad85424e809273b9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:23.210487 16242 raft_consensus.cc:2454] T 85733dae67104b429b2126a77c61d98d P 207b2c95ee544e16bafb3691d331f58e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4f919c590a94cf7b6f699e829e6d86b in term 1.
I20260812 03:12:23.210487 16379 raft_consensus.cc:2454] T 85733dae67104b429b2126a77c61d98d P ff15f90157e14440ad85424e809273b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4f919c590a94cf7b6f699e829e6d86b in term 1.
I20260812 03:12:23.210843 16454 leader_election.cc:302] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f4f919c590a94cf7b6f699e829e6d86b, ff15f90157e14440ad85424e809273b9; no voters:
I20260812 03:12:23.211023 16602 raft_consensus.cc:2781] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:23.211267 16602 raft_consensus.cc:686] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b [term 1 LEADER]: Becoming Leader. State: Replica: f4f919c590a94cf7b6f699e829e6d86b, State: Running, Role: LEADER
I20260812 03:12:23.211647 16602 consensus_queue.cc:237] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff15f90157e14440ad85424e809273b9" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41139 } } peers { permanent_uuid: "207b2c95ee544e16bafb3691d331f58e" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 36225 } } peers { permanent_uuid: "f4f919c590a94cf7b6f699e829e6d86b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39891 } }
I20260812 03:12:23.215049 16087 catalog_manager.cc:5495] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b reported cstate change: term changed from 0 to 1, leader changed from <none> to f4f919c590a94cf7b6f699e829e6d86b (127.26.250.131). New cstate: current_term: 1 leader_uuid: "f4f919c590a94cf7b6f699e829e6d86b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff15f90157e14440ad85424e809273b9" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "207b2c95ee544e16bafb3691d331f58e" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 36225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4f919c590a94cf7b6f699e829e6d86b" member_type: VOTER last_known_addr { host: "127.26.250.131" port: 39891 } health_report { overall_health: HEALTHY } } }
I20260812 03:12:23.248782 16590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:23.244968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60074 (local address 127.26.250.131:39891)
0812 03:12:23.245122 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:23.245127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:23.245143 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:12:23.245257 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:23.245260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:23.245319 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:23.245449 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:12:23.245719 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.246241 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.247178 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:23.247431 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:23.247559 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:23.247902 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:12:23.248349 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:12:23.248498 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:23.248565 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 03:12:23.248622 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260812 03:12:23.251396 16576 tablet.cc:2369] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:12:23.255445 16379 raft_consensus.cc:1261] T 85733dae67104b429b2126a77c61d98d P ff15f90157e14440ad85424e809273b9 [term 1 FOLLOWER]: Refusing update from remote peer f4f919c590a94cf7b6f699e829e6d86b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:12:23.255462 16242 raft_consensus.cc:1261] T 85733dae67104b429b2126a77c61d98d P 207b2c95ee544e16bafb3691d331f58e [term 1 FOLLOWER]: Refusing update from remote peer f4f919c590a94cf7b6f699e829e6d86b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:12:23.256410 16602 consensus_queue.cc:1035] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b [LEADER]: Connected to new peer: Peer: permanent_uuid: "207b2c95ee544e16bafb3691d331f58e" member_type: VOTER last_known_addr { host: "127.26.250.129" port: 36225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:12:23.256685 16605 consensus_queue.cc:1035] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff15f90157e14440ad85424e809273b9" member_type: VOTER last_known_addr { host: "127.26.250.130" port: 41139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:12:23.267030 16609 mvcc.cc:205] Tried to move back new op lower bound from 7317521789965283328 to 7317521789797761024. Current Snapshot: MvccSnapshot[applied={T|T < 7317521789965283328}]
I20260812 03:12:23.268153 16610 mvcc.cc:205] Tried to move back new op lower bound from 7317521789965283328 to 7317521789797761024. Current Snapshot: MvccSnapshot[applied={T|T < 7317521789965283328 or (T in {7317521789965283328})}]
I20260812 03:12:23.276513 16087 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:12:23.276716 16087 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 03:12:23.279270 16087 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f9f967c9fc214d779376874b730cb6d3: Sending DeleteTablet for 3 replicas of tablet 85733dae67104b429b2126a77c61d98d
I20260812 03:12:23.280227 16504 tablet_service.cc:1514] Processing DeleteTablet for tablet 85733dae67104b429b2126a77c61d98d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:12:23 UTC) from {username='slave'} at 127.0.0.1:60062
I20260812 03:12:23.280510 16359 tablet_service.cc:1514] Processing DeleteTablet for tablet 85733dae67104b429b2126a77c61d98d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:12:23 UTC) from {username='slave'} at 127.0.0.1:54468
I20260812 03:12:23.280778 16618 tablet_replica.cc:330] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b: stopping tablet replica
I20260812 03:12:23.281071 16619 tablet_replica.cc:330] T 85733dae67104b429b2126a77c61d98d P ff15f90157e14440ad85424e809273b9: stopping tablet replica
I20260812 03:12:23.281059 16222 tablet_service.cc:1514] Processing DeleteTablet for tablet 85733dae67104b429b2126a77c61d98d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 03:12:23 UTC) from {username='slave'} at 127.0.0.1:53520
I20260812 03:12:23.281180 16618 raft_consensus.cc:2229] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b [term 1 LEADER]: Raft consensus shutting down.
I20260812 03:12:23.281508 16619 raft_consensus.cc:2229] T 85733dae67104b429b2126a77c61d98d P ff15f90157e14440ad85424e809273b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 03:12:23.281637 16618 raft_consensus.cc:2258] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:12:23.281702 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 16157
I20260812 03:12:23.283245 16619 raft_consensus.cc:2258] T 85733dae67104b429b2126a77c61d98d P ff15f90157e14440ad85424e809273b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 03:12:23.285158 16619 ts_tablet_manager.cc:1905] T 85733dae67104b429b2126a77c61d98d P ff15f90157e14440ad85424e809273b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:12:23.285697 16618 ts_tablet_manager.cc:1905] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 03:12:23.288437 16619 ts_tablet_manager.cc:1918] T 85733dae67104b429b2126a77c61d98d P ff15f90157e14440ad85424e809273b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:12:23.288617 16618 ts_tablet_manager.cc:1918] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 03:12:23.288652 16619 log.cc:1196] T 85733dae67104b429b2126a77c61d98d P ff15f90157e14440ad85424e809273b9: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-1/wal/wals/85733dae67104b429b2126a77c61d98d
I20260812 03:12:23.289180 16619 ts_tablet_manager.cc:1939] T 85733dae67104b429b2126a77c61d98d P ff15f90157e14440ad85424e809273b9: Deleting consensus metadata
I20260812 03:12:23.291198 16072 catalog_manager.cc:4841] TS ff15f90157e14440ad85424e809273b9 (127.26.250.130:41139): tablet 85733dae67104b429b2126a77c61d98d (table test-table [id=c0310d5c5cda4382bf9128e292f775f9]) successfully deleted
I20260812 03:12:23.295676 16618 log.cc:1196] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b: Deleting WAL directory at /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786504075277362-27626-0/minicluster-data/ts-2/wal/wals/85733dae67104b429b2126a77c61d98d
W20260812 03:12:23.295666 16072 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.26.250.129:36225 (error 108)
I20260812 03:12:23.296260 16618 ts_tablet_manager.cc:1939] T 85733dae67104b429b2126a77c61d98d P f4f919c590a94cf7b6f699e829e6d86b: Deleting consensus metadata
I20260812 03:12:23.297289 16589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:23.296969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.129:36225 (local address 127.0.0.1:53536)
0812 03:12:23.297099 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:23.297186 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.129:36225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 03:12:23.297281 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 16294
I20260812 03:12:23.299177 16070 catalog_manager.cc:4841] TS f4f919c590a94cf7b6f699e829e6d86b (127.26.250.131:39891): tablet 85733dae67104b429b2126a77c61d98d (table test-table [id=c0310d5c5cda4382bf9128e292f775f9]) successfully deleted
W20260812 03:12:23.304523 16072 catalog_manager.cc:4584] TS 207b2c95ee544e16bafb3691d331f58e (127.26.250.129:36225): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 85733dae67104b429b2126a77c61d98d: Network error: Client connection negotiation failed: client connection to 127.26.250.129:36225: connect: Connection refused (error 111)
I20260812 03:12:23.312815 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 16435
I20260812 03:12:23.321741 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 16053
2026-08-12T03:12:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1257 ms)
[----------] 6 tests from EncryptionPolicyTest (11810 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:12:23 dist-test-slave-zpfg krb5kdc[16624](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:12:23 dist-test-slave-zpfg krb5kdc[16624](info): set up 2 sockets
Aug 12 03:12:23 dist-test-slave-zpfg krb5kdc[16624](info): commencing operation
krb5kdc: starting...
W20260812 03:12:24.435031 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:12:24 dist-test-slave-zpfg krb5kdc[16624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:12:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:12:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:12:24.578457 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35877
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:24.704094 16637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:24.704412 16637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:24.704463 16637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:24.708698 16637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:24.708770 16637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:24.708796 16637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:24.708813 16637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:24.708829 16637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:24.714432 16637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35877
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:24.715575 16637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:24.716620 16637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:24.725908 16642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:24.725878 16643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:24.725943 16646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:24.726426 16637 server_base.cc:1034] running on GCE node
I20260812 03:12:24.727048 16637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:24.728245 16637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:24.729449 16637 hybrid_clock.cc:648] HybridClock initialized: now 1786504344729412 us; error 54 us; skew 500 ppm
Aug 12 03:12:24 dist-test-slave-zpfg krb5kdc[16624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504344, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:24.732654 16637 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:12:24.733698 16637 webserver.cc:460] Webserver started at http://127.0.0.1:45667/ using document root <none> and password file <none>
I20260812 03:12:24.734295 16637 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:24.734457 16637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:24.734709 16637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:24.736905 16637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "24554759d8f74c75ad7efbba3e86a09a"
format_stamp: "Formatted at 2026-08-12 03:12:24 on dist-test-slave-zpfg"
server_key: "47d8dfe8892c04166c83bef35e1545c0"
server_key_iv: "2c7c711d2ffbc2b1ad826dd3f8b68a92"
server_key_version: "encryptionkey@0"
I20260812 03:12:24.737522 16637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "24554759d8f74c75ad7efbba3e86a09a"
format_stamp: "Formatted at 2026-08-12 03:12:24 on dist-test-slave-zpfg"
server_key: "47d8dfe8892c04166c83bef35e1545c0"
server_key_iv: "2c7c711d2ffbc2b1ad826dd3f8b68a92"
server_key_version: "encryptionkey@0"
I20260812 03:12:24.742095 16637 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:12:24.744810 16653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:24.746198 16637 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:12:24.746470 16637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "24554759d8f74c75ad7efbba3e86a09a"
format_stamp: "Formatted at 2026-08-12 03:12:24 on dist-test-slave-zpfg"
server_key: "47d8dfe8892c04166c83bef35e1545c0"
server_key_iv: "2c7c711d2ffbc2b1ad826dd3f8b68a92"
server_key_version: "encryptionkey@0"
I20260812 03:12:24.746582 16637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:24.771204 16637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:24.775830 16637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:24.776266 16637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:24.786588 16637 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35877
I20260812 03:12:24.786897 16710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35877 every 8 connection(s)
I20260812 03:12:24.787738 16637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:12:24.790899 16711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:24.797089 16711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24554759d8f74c75ad7efbba3e86a09a: Bootstrap starting.
I20260812 03:12:24.797475 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 16637
I20260812 03:12:24.797632 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:12:24.797955 27626 external_mini_cluster.cc:1428] Setting key 6df2f5c2a3062e3c46a994d9743f6fea
I20260812 03:12:24.800170 16711 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24554759d8f74c75ad7efbba3e86a09a: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:24.800938 16711 log.cc:824] T 00000000000000000000000000000000 P 24554759d8f74c75ad7efbba3e86a09a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:24.803103 16711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24554759d8f74c75ad7efbba3e86a09a: No bootstrap required, opened a new log
I20260812 03:12:24.806093 16711 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24554759d8f74c75ad7efbba3e86a09a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24554759d8f74c75ad7efbba3e86a09a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35877 } }
I20260812 03:12:24.806348 16711 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24554759d8f74c75ad7efbba3e86a09a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:24.806425 16711 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24554759d8f74c75ad7efbba3e86a09a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24554759d8f74c75ad7efbba3e86a09a, State: Initialized, Role: FOLLOWER
I20260812 03:12:24.806967 16711 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24554759d8f74c75ad7efbba3e86a09a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24554759d8f74c75ad7efbba3e86a09a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35877 } }
I20260812 03:12:24.807176 16711 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24554759d8f74c75ad7efbba3e86a09a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:12:24.807271 16711 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24554759d8f74c75ad7efbba3e86a09a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:12:24.807374 16711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24554759d8f74c75ad7efbba3e86a09a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:24.808331 16711 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24554759d8f74c75ad7efbba3e86a09a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24554759d8f74c75ad7efbba3e86a09a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35877 } }
Aug 12 03:12:24 dist-test-slave-zpfg krb5kdc[16624](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 03:12:24.808707 16711 leader_election.cc:302] T 00000000000000000000000000000000 P 24554759d8f74c75ad7efbba3e86a09a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24554759d8f74c75ad7efbba3e86a09a; no voters:
I20260812 03:12:24.809073 16711 leader_election.cc:288] T 00000000000000000000000000000000 P 24554759d8f74c75ad7efbba3e86a09a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:12:24.809195 16717 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24554759d8f74c75ad7efbba3e86a09a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:24.809424 16717 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24554759d8f74c75ad7efbba3e86a09a [term 1 LEADER]: Becoming Leader. State: Replica: 24554759d8f74c75ad7efbba3e86a09a, State: Running, Role: LEADER
I20260812 03:12:24.809748 16717 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24554759d8f74c75ad7efbba3e86a09a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24554759d8f74c75ad7efbba3e86a09a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35877 } }
I20260812 03:12:24.810525 16711 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24554759d8f74c75ad7efbba3e86a09a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:24.812084 16719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24554759d8f74c75ad7efbba3e86a09a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24554759d8f74c75ad7efbba3e86a09a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24554759d8f74c75ad7efbba3e86a09a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35877 } } }
I20260812 03:12:24.812247 16719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24554759d8f74c75ad7efbba3e86a09a [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:24.812498 16719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24554759d8f74c75ad7efbba3e86a09a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24554759d8f74c75ad7efbba3e86a09a. Latest consensus state: current_term: 1 leader_uuid: "24554759d8f74c75ad7efbba3e86a09a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24554759d8f74c75ad7efbba3e86a09a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35877 } } }
I20260812 03:12:24.812593 16719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24554759d8f74c75ad7efbba3e86a09a [sys.catalog]: This master's current role is: LEADER
W20260812 03:12:24.815923 16733 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 24554759d8f74c75ad7efbba3e86a09a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:12:24.816027 16733 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:12:24.816097 16730 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:12:24.816407 16715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:24.799583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35235 (local address 127.0.0.1:35877)
0812 03:12:24.800079 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:24.800096 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:24.800136 (+ 40us) server_negotiation.cc:407] Connection header received
0812 03:12:24.800915 (+ 779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:24.800934 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:24.801384 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:24.801751 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:24.802823 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:24.803974 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:24.804955 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:24.805983 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:24.806172 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:24.809570 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:24.809581 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:24.809601 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:24.809633 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:24.811878 (+ 2245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:24.812618 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:24.812627 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:24.812636 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:24.812796 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:24.813389 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:24.813391 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:24.813393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:24.813847 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:24.814124 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:24.815034 (+ 910us) server_negotiation.cc:299] Negotiation successful
0812 03:12:24.815298 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":158,"threads_started":1}
I20260812 03:12:24.816958 16730 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:12:24.819896 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:12:24.823864 16730 catalog_manager.cc:1347] Generated new cluster ID: 3df9c65d5c0b4864b42c35a507bd20cf
I20260812 03:12:24.823951 16730 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:12:24.830762 16715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:24.822066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33173 (local address 127.0.0.1:35877)
0812 03:12:24.822205 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:24.822211 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:24.822517 (+ 306us) server_negotiation.cc:407] Connection header received
0812 03:12:24.822654 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:24.822658 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:24.822723 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:24.822828 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:24.823687 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:24.824441 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:24.825307 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:24.825570 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:24.825688 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:24.827170 (+ 1482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:24.827174 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:24.827176 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:24.827203 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:24.828777 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:24.829310 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:24.829316 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:24.829318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:24.829408 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:24.829767 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:24.829770 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:24.829772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:24.829956 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:24.830119 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:24.830429 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 03:12:24.830533 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:12:24.879245 16730 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:12:24.880507 16730 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:12:24.885103 16730 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24554759d8f74c75ad7efbba3e86a09a: Generated new TSK 0
I20260812 03:12:24.886077 16730 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:12:24.898707 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 16637
W20260812 03:12:24.907555 16622 connection.cc:535] client connection to 127.0.0.1:35877 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35877): Connection reset by peer (error 104)
2026-08-12T03:12:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1575 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:12:24 dist-test-slave-zpfg krb5kdc[16747](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:12:24 dist-test-slave-zpfg krb5kdc[16747](info): set up 2 sockets
Aug 12 03:12:24 dist-test-slave-zpfg krb5kdc[16747](info): commencing operation
krb5kdc: starting...
W20260812 03:12:26.007773 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:12:26 dist-test-slave-zpfg krb5kdc[16747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:12:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:12:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:12:26.151728 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34311
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:26.272886 16762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:26.273237 16762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:26.273321 16762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:26.277572 16762 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:26.277696 16762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:26.277735 16762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:26.277784 16762 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:26.277820 16762 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:26.283468 16762 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34311
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:26.284801 16762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:26.285956 16762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:26.295410 16772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:26.295432 16768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:26.295499 16762 server_base.cc:1034] running on GCE node
W20260812 03:12:26.295432 16770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:26.296339 16762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:26.297675 16762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:26.298884 16762 hybrid_clock.cc:648] HybridClock initialized: now 1786504346298864 us; error 37 us; skew 500 ppm
Aug 12 03:12:26 dist-test-slave-zpfg krb5kdc[16747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504346, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:26.307801 16762 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:12:26.308951 16762 webserver.cc:460] Webserver started at http://127.0.0.1:40997/ using document root <none> and password file <none>
I20260812 03:12:26.309537 16762 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:26.309603 16762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:26.309785 16762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:26.312227 16762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "74cb01101a2a4de1aa27e5f8f3cf95a9"
format_stamp: "Formatted at 2026-08-12 03:12:26 on dist-test-slave-zpfg"
server_key: "dfa8e52086b4b1ac48c267f17c8dbb13"
server_key_iv: "56b047dbc297eb72326f91183021d992"
server_key_version: "encryptionkey@0"
I20260812 03:12:26.312835 16762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "74cb01101a2a4de1aa27e5f8f3cf95a9"
format_stamp: "Formatted at 2026-08-12 03:12:26 on dist-test-slave-zpfg"
server_key: "dfa8e52086b4b1ac48c267f17c8dbb13"
server_key_iv: "56b047dbc297eb72326f91183021d992"
server_key_version: "encryptionkey@0"
I20260812 03:12:26.317194 16762 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 03:12:26.320305 16780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:26.321980 16762 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:12:26.322118 16762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "74cb01101a2a4de1aa27e5f8f3cf95a9"
format_stamp: "Formatted at 2026-08-12 03:12:26 on dist-test-slave-zpfg"
server_key: "dfa8e52086b4b1ac48c267f17c8dbb13"
server_key_iv: "56b047dbc297eb72326f91183021d992"
server_key_version: "encryptionkey@0"
I20260812 03:12:26.322260 16762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:26.344452 16762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:26.348418 16762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:26.348620 16762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:26.357388 16762 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34311
I20260812 03:12:26.357456 16834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34311 every 8 connection(s)
I20260812 03:12:26.358536 16762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:12:26.360457 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 16762
I20260812 03:12:26.360605 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:12:26.360949 27626 external_mini_cluster.cc:1428] Setting key f582cf0aac9e9b8662e84ddb56a79139
I20260812 03:12:26.361896 16835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:26.368147 16835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74cb01101a2a4de1aa27e5f8f3cf95a9: Bootstrap starting.
I20260812 03:12:26.370877 16835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74cb01101a2a4de1aa27e5f8f3cf95a9: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:26.370970 16838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:26.362552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54747 (local address 127.0.0.1:34311)
0812 03:12:26.363023 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:26.363039 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:26.363078 (+ 39us) server_negotiation.cc:407] Connection header received
0812 03:12:26.363790 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:26.363808 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:26.364159 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:26.364513 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:26.364967 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:26.366030 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:26.367045 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:26.367544 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:26.367732 (+ 188us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:26.368850 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:26.368856 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:26.368871 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:26.369306 (+ 435us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:26.369369 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:26.369394 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:26.369605 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:26.369869 (+ 264us) server_negotiation.cc:299] Negotiation successful
0812 03:12:26.370046 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":158,"threads_started":1}
I20260812 03:12:26.371878 16835 log.cc:824] T 00000000000000000000000000000000 P 74cb01101a2a4de1aa27e5f8f3cf95a9: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:26.374152 16835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74cb01101a2a4de1aa27e5f8f3cf95a9: No bootstrap required, opened a new log
I20260812 03:12:26.374583 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:12:26.378649 16835 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74cb01101a2a4de1aa27e5f8f3cf95a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74cb01101a2a4de1aa27e5f8f3cf95a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34311 } }
I20260812 03:12:26.378979 16835 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74cb01101a2a4de1aa27e5f8f3cf95a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:26.379050 16835 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74cb01101a2a4de1aa27e5f8f3cf95a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74cb01101a2a4de1aa27e5f8f3cf95a9, State: Initialized, Role: FOLLOWER
I20260812 03:12:26.379555 16835 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74cb01101a2a4de1aa27e5f8f3cf95a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74cb01101a2a4de1aa27e5f8f3cf95a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34311 } }
I20260812 03:12:26.379738 16835 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74cb01101a2a4de1aa27e5f8f3cf95a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:12:26.379802 16835 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74cb01101a2a4de1aa27e5f8f3cf95a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:12:26.379922 16835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74cb01101a2a4de1aa27e5f8f3cf95a9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:26.381301 16835 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74cb01101a2a4de1aa27e5f8f3cf95a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74cb01101a2a4de1aa27e5f8f3cf95a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34311 } }
I20260812 03:12:26.381699 16835 leader_election.cc:302] T 00000000000000000000000000000000 P 74cb01101a2a4de1aa27e5f8f3cf95a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74cb01101a2a4de1aa27e5f8f3cf95a9; no voters:
I20260812 03:12:26.381929 16838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:26.377020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57787 (local address 127.0.0.1:34311)
0812 03:12:26.377173 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:26.377177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:26.377409 (+ 232us) server_negotiation.cc:407] Connection header received
0812 03:12:26.377540 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:26.377542 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:26.377584 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:26.377659 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:26.377990 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:26.378754 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:26.379946 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:26.380269 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:26.380376 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:26.381329 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:26.381333 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:26.381336 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:26.381351 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:26.381376 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:26.381413 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:26.381518 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:26.381685 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 03:12:26.381727 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":60}
I20260812 03:12:26.382150 16835 leader_election.cc:288] T 00000000000000000000000000000000 P 74cb01101a2a4de1aa27e5f8f3cf95a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:12:26.382252 16845 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74cb01101a2a4de1aa27e5f8f3cf95a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:26.382555 16845 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74cb01101a2a4de1aa27e5f8f3cf95a9 [term 1 LEADER]: Becoming Leader. State: Replica: 74cb01101a2a4de1aa27e5f8f3cf95a9, State: Running, Role: LEADER
I20260812 03:12:26.382970 16845 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74cb01101a2a4de1aa27e5f8f3cf95a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74cb01101a2a4de1aa27e5f8f3cf95a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34311 } }
I20260812 03:12:26.383599 16835 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74cb01101a2a4de1aa27e5f8f3cf95a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:26.384342 16845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74cb01101a2a4de1aa27e5f8f3cf95a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74cb01101a2a4de1aa27e5f8f3cf95a9. Latest consensus state: current_term: 1 leader_uuid: "74cb01101a2a4de1aa27e5f8f3cf95a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74cb01101a2a4de1aa27e5f8f3cf95a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34311 } } }
I20260812 03:12:26.384613 16845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74cb01101a2a4de1aa27e5f8f3cf95a9 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:26.384784 16846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74cb01101a2a4de1aa27e5f8f3cf95a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74cb01101a2a4de1aa27e5f8f3cf95a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74cb01101a2a4de1aa27e5f8f3cf95a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34311 } } }
I20260812 03:12:26.384960 16846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74cb01101a2a4de1aa27e5f8f3cf95a9 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:26.385097 16854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:12:26.388680 16854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:12:26.395764 16854 catalog_manager.cc:1347] Generated new cluster ID: efd1b57c93204f308251dae1270277e8
I20260812 03:12:26.395880 16854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:12:26.417454 16854 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:12:26.418700 16854 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:12:26.435403 16854 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74cb01101a2a4de1aa27e5f8f3cf95a9: Generated new TSK 0
I20260812 03:12:26.436282 16854 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:12:26.443552 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 16762
W20260812 03:12:26.452956 16745 connection.cc:535] client connection to 127.0.0.1:34311 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34311): Connection reset by peer (error 104)
2026-08-12T03:12:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1545 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:12:26 dist-test-slave-zpfg krb5kdc[16867](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:12:26 dist-test-slave-zpfg krb5kdc[16867](info): set up 2 sockets
Aug 12 03:12:26 dist-test-slave-zpfg krb5kdc[16867](info): commencing operation
krb5kdc: starting...
W20260812 03:12:27.579756 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:12:27 dist-test-slave-zpfg krb5kdc[16867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:12:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:12:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:12:27.722596 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39813
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:27.844475 16881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:27.844848 16881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:27.844935 16881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:27.849422 16881 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:27.849557 16881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:27.849597 16881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:27.849643 16881 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:27.849676 16881 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:27.855589 16881 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39813
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:27.856849 16881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:27.857968 16881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:27.867861 16886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:27.867865 16889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:27.867887 16887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:27.868136 16881 server_base.cc:1034] running on GCE node
I20260812 03:12:27.868892 16881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:27.870232 16881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:27.871471 16881 hybrid_clock.cc:648] HybridClock initialized: now 1786504347871454 us; error 32 us; skew 500 ppm
Aug 12 03:12:27 dist-test-slave-zpfg krb5kdc[16867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504347, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:27.875029 16881 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:12:27.876150 16881 webserver.cc:460] Webserver started at http://127.0.0.1:32863/ using document root <none> and password file <none>
I20260812 03:12:27.876760 16881 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:27.876861 16881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:27.877107 16881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:27.879351 16881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "ea09936de6294d3485ec2749118e63f7"
format_stamp: "Formatted at 2026-08-12 03:12:27 on dist-test-slave-zpfg"
server_key: "4a1601b52155cb0853d99feb1b5c5139"
server_key_iv: "75f312db6c46682c9044f92c581b21fc"
server_key_version: "encryptionkey@0"
I20260812 03:12:27.879945 16881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "ea09936de6294d3485ec2749118e63f7"
format_stamp: "Formatted at 2026-08-12 03:12:27 on dist-test-slave-zpfg"
server_key: "4a1601b52155cb0853d99feb1b5c5139"
server_key_iv: "75f312db6c46682c9044f92c581b21fc"
server_key_version: "encryptionkey@0"
I20260812 03:12:27.884126 16881 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260812 03:12:27.887049 16898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:27.888578 16881 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:12:27.888770 16881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "ea09936de6294d3485ec2749118e63f7"
format_stamp: "Formatted at 2026-08-12 03:12:27 on dist-test-slave-zpfg"
server_key: "4a1601b52155cb0853d99feb1b5c5139"
server_key_iv: "75f312db6c46682c9044f92c581b21fc"
server_key_version: "encryptionkey@0"
I20260812 03:12:27.888921 16881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:27.894058 16881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:27.897881 16881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:27.898123 16881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:27.906066 16881 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39813
I20260812 03:12:27.906159 16955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39813 every 8 connection(s)
I20260812 03:12:27.907215 16881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:12:27.910660 16956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:27.911032 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 16881
I20260812 03:12:27.911193 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:12:27.911573 27626 external_mini_cluster.cc:1428] Setting key 603c2b9f0b7fe12279f3b5c131767b13
I20260812 03:12:27.917321 16959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:27.913152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53437 (local address 127.0.0.1:39813)
0812 03:12:27.913603 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:27.913618 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:27.913660 (+ 42us) server_negotiation.cc:407] Connection header received
0812 03:12:27.914436 (+ 776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:27.914458 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:27.914767 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:27.915135 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:27.915996 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:27.916000 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:27.916013 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:27.916023 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:27.916082 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:27.916102 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:27.916283 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:27.916363 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 03:12:27.916536 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":137,"threads_started":1}
I20260812 03:12:27.919495 16956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea09936de6294d3485ec2749118e63f7: Bootstrap starting.
I20260812 03:12:27.920441 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:12:27.922098 16956 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea09936de6294d3485ec2749118e63f7: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:27.923111 16956 log.cc:824] T 00000000000000000000000000000000 P ea09936de6294d3485ec2749118e63f7: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:27.925354 16959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:27.922705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60113 (local address 127.0.0.1:39813)
0812 03:12:27.922845 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:27.922850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:27.923772 (+ 922us) server_negotiation.cc:407] Connection header received
0812 03:12:27.923935 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:27.923939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:27.923982 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:27.924076 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:27.924877 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:27.924881 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:27.924884 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:27.924896 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:27.924932 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:27.924943 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:27.925087 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:27.925183 (+ 96us) server_negotiation.cc:299] Negotiation successful
0812 03:12:27.925225 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 03:12:27.925488 16956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea09936de6294d3485ec2749118e63f7: No bootstrap required, opened a new log
I20260812 03:12:27.929494 16956 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea09936de6294d3485ec2749118e63f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea09936de6294d3485ec2749118e63f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39813 } }
I20260812 03:12:27.929811 16956 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea09936de6294d3485ec2749118e63f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:27.929916 16956 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea09936de6294d3485ec2749118e63f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea09936de6294d3485ec2749118e63f7, State: Initialized, Role: FOLLOWER
I20260812 03:12:27.930536 16956 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea09936de6294d3485ec2749118e63f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea09936de6294d3485ec2749118e63f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39813 } }
I20260812 03:12:27.930748 16956 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea09936de6294d3485ec2749118e63f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:12:27.930848 16956 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea09936de6294d3485ec2749118e63f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:12:27.930955 16956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea09936de6294d3485ec2749118e63f7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:27.932224 16956 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea09936de6294d3485ec2749118e63f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea09936de6294d3485ec2749118e63f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39813 } }
I20260812 03:12:27.932605 16956 leader_election.cc:302] T 00000000000000000000000000000000 P ea09936de6294d3485ec2749118e63f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea09936de6294d3485ec2749118e63f7; no voters:
I20260812 03:12:27.932976 16956 leader_election.cc:288] T 00000000000000000000000000000000 P ea09936de6294d3485ec2749118e63f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:12:27.933135 16966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea09936de6294d3485ec2749118e63f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:27.933409 16966 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea09936de6294d3485ec2749118e63f7 [term 1 LEADER]: Becoming Leader. State: Replica: ea09936de6294d3485ec2749118e63f7, State: Running, Role: LEADER
I20260812 03:12:27.933736 16966 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea09936de6294d3485ec2749118e63f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea09936de6294d3485ec2749118e63f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39813 } }
I20260812 03:12:27.934391 16956 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea09936de6294d3485ec2749118e63f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:27.935127 16968 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea09936de6294d3485ec2749118e63f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea09936de6294d3485ec2749118e63f7. Latest consensus state: current_term: 1 leader_uuid: "ea09936de6294d3485ec2749118e63f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea09936de6294d3485ec2749118e63f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39813 } } }
I20260812 03:12:27.935262 16968 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea09936de6294d3485ec2749118e63f7 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:27.935578 16967 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea09936de6294d3485ec2749118e63f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea09936de6294d3485ec2749118e63f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea09936de6294d3485ec2749118e63f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39813 } } }
I20260812 03:12:27.935683 16967 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea09936de6294d3485ec2749118e63f7 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:27.935770 16973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:12:27.939647 16973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:12:27.946260 16973 catalog_manager.cc:1347] Generated new cluster ID: e1ac3510d15c4eefa88901e20b6176a2
I20260812 03:12:27.946376 16973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:12:27.980978 16973 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:12:27.981449 16973 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:12:27.989007 16973 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea09936de6294d3485ec2749118e63f7: Generated new TSK 0
I20260812 03:12:27.989900 16973 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:12:27.993954 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 16881
2026-08-12T03:12:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1550 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:12:28 dist-test-slave-zpfg krb5kdc[16987](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:12:28 dist-test-slave-zpfg krb5kdc[16987](info): set up 2 sockets
Aug 12 03:12:28 dist-test-slave-zpfg krb5kdc[16987](info): commencing operation
krb5kdc: starting...
W20260812 03:12:29.091265 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:12:29 dist-test-slave-zpfg krb5kdc[16987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:12:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:12:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:12:29.238224 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42065
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:29.366562 17001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:29.366851 17001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:29.366896 17001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:29.371061 17001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:29.371131 17001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:29.371152 17001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:29.371169 17001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:29.371186 17001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:29.376869 17001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:29.378036 17001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:29.379168 17001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:29.389217 17010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:29.389226 17008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:29.389465 17001 server_base.cc:1034] running on GCE node
W20260812 03:12:29.389312 17007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:29.390329 17001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:29.391721 17001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:29.392942 17001 hybrid_clock.cc:648] HybridClock initialized: now 1786504349392919 us; error 38 us; skew 500 ppm
Aug 12 03:12:29 dist-test-slave-zpfg krb5kdc[16987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504349, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:29.396509 17001 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:12:29.397583 17001 webserver.cc:460] Webserver started at http://127.0.0.1:38459/ using document root <none> and password file <none>
I20260812 03:12:29.398293 17001 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:29.398428 17001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:29.398706 17001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:29.401064 17001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "f20750bedb1c4220bd232f8f2bd6bfd5"
format_stamp: "Formatted at 2026-08-12 03:12:29 on dist-test-slave-zpfg"
server_key: "3018396666765efc57fae8f429e98559"
server_key_iv: "9a36c9b70a3cec95374dfa6265538c41"
server_key_version: "encryptionkey@0"
I20260812 03:12:29.401741 17001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "f20750bedb1c4220bd232f8f2bd6bfd5"
format_stamp: "Formatted at 2026-08-12 03:12:29 on dist-test-slave-zpfg"
server_key: "3018396666765efc57fae8f429e98559"
server_key_iv: "9a36c9b70a3cec95374dfa6265538c41"
server_key_version: "encryptionkey@0"
I20260812 03:12:29.406247 17001 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260812 03:12:29.409462 17018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:29.410998 17001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:12:29.411230 17001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "f20750bedb1c4220bd232f8f2bd6bfd5"
format_stamp: "Formatted at 2026-08-12 03:12:29 on dist-test-slave-zpfg"
server_key: "3018396666765efc57fae8f429e98559"
server_key_iv: "9a36c9b70a3cec95374dfa6265538c41"
server_key_version: "encryptionkey@0"
I20260812 03:12:29.411372 17001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:29.425283 17001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:29.428934 17001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:29.429170 17001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:29.436874 17001 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42065
I20260812 03:12:29.436903 17079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42065 every 8 connection(s)
I20260812 03:12:29.437870 17001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:12:29.441146 17080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:29.447944 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 17001
I20260812 03:12:29.448006 17080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f20750bedb1c4220bd232f8f2bd6bfd5: Bootstrap starting.
I20260812 03:12:29.448122 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:12:29.448468 27626 external_mini_cluster.cc:1428] Setting key 1a32134c4c5c74d67dd0c2de03c3af73
I20260812 03:12:29.451108 17080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f20750bedb1c4220bd232f8f2bd6bfd5: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:29.451956 17080 log.cc:824] T 00000000000000000000000000000000 P f20750bedb1c4220bd232f8f2bd6bfd5: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:29.454471 17080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f20750bedb1c4220bd232f8f2bd6bfd5: No bootstrap required, opened a new log
I20260812 03:12:29.459482 17080 raft_consensus.cc:348] T 00000000000000000000000000000000 P f20750bedb1c4220bd232f8f2bd6bfd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f20750bedb1c4220bd232f8f2bd6bfd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42065 } }
I20260812 03:12:29.459748 17080 raft_consensus.cc:374] T 00000000000000000000000000000000 P f20750bedb1c4220bd232f8f2bd6bfd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
Aug 12 03:12:29 dist-test-slave-zpfg krb5kdc[16987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 03:12:29.459789 17080 raft_consensus.cc:729] T 00000000000000000000000000000000 P f20750bedb1c4220bd232f8f2bd6bfd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f20750bedb1c4220bd232f8f2bd6bfd5, State: Initialized, Role: FOLLOWER
I20260812 03:12:29.460327 17080 consensus_queue.cc:260] T 00000000000000000000000000000000 P f20750bedb1c4220bd232f8f2bd6bfd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f20750bedb1c4220bd232f8f2bd6bfd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42065 } }
I20260812 03:12:29.460506 17080 raft_consensus.cc:388] T 00000000000000000000000000000000 P f20750bedb1c4220bd232f8f2bd6bfd5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:12:29.460572 17080 raft_consensus.cc:482] T 00000000000000000000000000000000 P f20750bedb1c4220bd232f8f2bd6bfd5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:12:29.460700 17080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f20750bedb1c4220bd232f8f2bd6bfd5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:29.462095 17080 raft_consensus.cc:504] T 00000000000000000000000000000000 P f20750bedb1c4220bd232f8f2bd6bfd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f20750bedb1c4220bd232f8f2bd6bfd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42065 } }
I20260812 03:12:29.462548 17080 leader_election.cc:302] T 00000000000000000000000000000000 P f20750bedb1c4220bd232f8f2bd6bfd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f20750bedb1c4220bd232f8f2bd6bfd5; no voters:
I20260812 03:12:29.462996 17080 leader_election.cc:288] T 00000000000000000000000000000000 P f20750bedb1c4220bd232f8f2bd6bfd5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:12:29.463249 17086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f20750bedb1c4220bd232f8f2bd6bfd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:29.463490 17086 raft_consensus.cc:686] T 00000000000000000000000000000000 P f20750bedb1c4220bd232f8f2bd6bfd5 [term 1 LEADER]: Becoming Leader. State: Replica: f20750bedb1c4220bd232f8f2bd6bfd5, State: Running, Role: LEADER
I20260812 03:12:29.463927 17086 consensus_queue.cc:237] T 00000000000000000000000000000000 P f20750bedb1c4220bd232f8f2bd6bfd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f20750bedb1c4220bd232f8f2bd6bfd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42065 } }
I20260812 03:12:29.464699 17080 sys_catalog.cc:564] T 00000000000000000000000000000000 P f20750bedb1c4220bd232f8f2bd6bfd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:29.465776 17088 sys_catalog.cc:455] T 00000000000000000000000000000000 P f20750bedb1c4220bd232f8f2bd6bfd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f20750bedb1c4220bd232f8f2bd6bfd5. Latest consensus state: current_term: 1 leader_uuid: "f20750bedb1c4220bd232f8f2bd6bfd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f20750bedb1c4220bd232f8f2bd6bfd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42065 } } }
I20260812 03:12:29.466223 17088 sys_catalog.cc:458] T 00000000000000000000000000000000 P f20750bedb1c4220bd232f8f2bd6bfd5 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:29.466104 17087 sys_catalog.cc:455] T 00000000000000000000000000000000 P f20750bedb1c4220bd232f8f2bd6bfd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f20750bedb1c4220bd232f8f2bd6bfd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f20750bedb1c4220bd232f8f2bd6bfd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42065 } } }
I20260812 03:12:29.466439 17087 sys_catalog.cc:458] T 00000000000000000000000000000000 P f20750bedb1c4220bd232f8f2bd6bfd5 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:29.466717 17084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:29.450236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56263 (local address 127.0.0.1:42065)
0812 03:12:29.450819 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:29.450835 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:29.450876 (+ 41us) server_negotiation.cc:407] Connection header received
0812 03:12:29.451785 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:29.451806 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:29.452235 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:29.452638 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:29.453604 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:29.455010 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:29.456368 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:29.456963 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:29.457111 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:29.460557 (+ 3446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:29.460564 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:29.460596 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:29.460639 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:29.463088 (+ 2449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:29.463673 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:29.463679 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:29.463686 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:29.463843 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:29.464166 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:29.464172 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:29.464174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:29.464602 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:29.464873 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:29.465221 (+ 348us) server_negotiation.cc:299] Negotiation successful
0812 03:12:29.465609 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":157,"threads_started":1}
I20260812 03:12:29.466995 17095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:12:29.470496 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:12:29.470530 17095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:12:29.476776 17095 catalog_manager.cc:1347] Generated new cluster ID: e354dffeabcd4515b8c4654f865fc38f
I20260812 03:12:29.476859 17095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:12:29.483544 17084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:29.473050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45375 (local address 127.0.0.1:42065)
0812 03:12:29.473177 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:29.473182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:29.474674 (+ 1492us) server_negotiation.cc:407] Connection header received
0812 03:12:29.474859 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:29.474862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:29.474944 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:29.475031 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:29.476065 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:29.476942 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:29.478032 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:29.478325 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:29.478497 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:29.480052 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:29.480056 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:29.480059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:29.480089 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:29.481658 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:29.482150 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:29.482153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:29.482155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:29.482244 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:29.482575 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:29.482579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:29.482581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:29.482779 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:29.482928 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:29.483191 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 03:12:29.483335 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:12:29.490981 17095 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:12:29.491967 17095 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:12:29.498306 17095 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f20750bedb1c4220bd232f8f2bd6bfd5: Generated new TSK 0
I20260812 03:12:29.499130 17095 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:12:29.504277 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 17001
2026-08-12T03:12:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1509 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:12:29 dist-test-slave-zpfg krb5kdc[17113](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:12:29 dist-test-slave-zpfg krb5kdc[17113](info): set up 2 sockets
Aug 12 03:12:29 dist-test-slave-zpfg krb5kdc[17113](info): commencing operation
krb5kdc: starting...
W20260812 03:12:30.619391 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:12:30 dist-test-slave-zpfg krb5kdc[17113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:12:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:12:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:12:30.757928 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35469
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35469
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:30.883551 17126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:30.883890 17126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:30.883977 17126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:30.888132 17126 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:30.888250 17126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:30.888288 17126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:30.888334 17126 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:30.888372 17126 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:30.893857 17126 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:30.895093 17126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:30.896229 17126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:30.905829 17134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:30.905833 17133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:30.905876 17136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:30.906133 17126 server_base.cc:1034] running on GCE node
I20260812 03:12:30.907027 17126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:30.908372 17126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:30.909611 17126 hybrid_clock.cc:648] HybridClock initialized: now 1786504350909575 us; error 53 us; skew 500 ppm
Aug 12 03:12:30 dist-test-slave-zpfg krb5kdc[17113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504350, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:30.913082 17126 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:12:30.914116 17126 webserver.cc:460] Webserver started at http://127.0.0.1:44933/ using document root <none> and password file <none>
I20260812 03:12:30.914793 17126 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:30.914901 17126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:30.915158 17126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:30.917438 17126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "272e859e780c4704aaea4c10ad30231f"
format_stamp: "Formatted at 2026-08-12 03:12:30 on dist-test-slave-zpfg"
server_key: "bccf8df05a9414f42cc94ce48b327287"
server_key_iv: "51cdab06c7874fdad09af6e8ad40cfec"
server_key_version: "encryptionkey@0"
I20260812 03:12:30.918079 17126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "272e859e780c4704aaea4c10ad30231f"
format_stamp: "Formatted at 2026-08-12 03:12:30 on dist-test-slave-zpfg"
server_key: "bccf8df05a9414f42cc94ce48b327287"
server_key_iv: "51cdab06c7874fdad09af6e8ad40cfec"
server_key_version: "encryptionkey@0"
I20260812 03:12:30.922505 17126 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 03:12:30.925509 17143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:30.927160 17126 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:12:30.927316 17126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "272e859e780c4704aaea4c10ad30231f"
format_stamp: "Formatted at 2026-08-12 03:12:30 on dist-test-slave-zpfg"
server_key: "bccf8df05a9414f42cc94ce48b327287"
server_key_iv: "51cdab06c7874fdad09af6e8ad40cfec"
server_key_version: "encryptionkey@0"
I20260812 03:12:30.927462 17126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:30.941701 17126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:30.945535 17126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:30.945763 17126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:30.953411 17126 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35469
I20260812 03:12:30.953441 17200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35469 every 8 connection(s)
I20260812 03:12:30.954469 17126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:12:30.956099 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 17126
I20260812 03:12:30.956264 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:12:30.956614 27626 external_mini_cluster.cc:1428] Setting key 96e5a7da70be3ede06e366cea11858ad
I20260812 03:12:30.957909 17201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:30.965461 17201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 272e859e780c4704aaea4c10ad30231f: Bootstrap starting.
I20260812 03:12:30.967597 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:30.958216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49181 (local address 127.0.0.1:35469)
0812 03:12:30.958713 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:30.958741 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:30.958787 (+ 46us) server_negotiation.cc:407] Connection header received
0812 03:12:30.959671 (+ 884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:30.959694 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:30.960127 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:30.960514 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:30.960946 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:30.962470 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:30.963581 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:30.964320 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:30.964477 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:30.965728 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:30.965734 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:30.965747 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:30.966161 (+ 414us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:30.966220 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:30.966245 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:30.966397 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:30.966600 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 03:12:30.966836 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":189,"threads_started":1}
I20260812 03:12:30.968125 17201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 272e859e780c4704aaea4c10ad30231f: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:30.968896 17201 log.cc:824] T 00000000000000000000000000000000 P 272e859e780c4704aaea4c10ad30231f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:30.970572 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:12:30.971143 17201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 272e859e780c4704aaea4c10ad30231f: No bootstrap required, opened a new log
I20260812 03:12:30.975016 17201 raft_consensus.cc:348] T 00000000000000000000000000000000 P 272e859e780c4704aaea4c10ad30231f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "272e859e780c4704aaea4c10ad30231f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35469 } }
I20260812 03:12:30.975281 17201 raft_consensus.cc:374] T 00000000000000000000000000000000 P 272e859e780c4704aaea4c10ad30231f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:30.975368 17201 raft_consensus.cc:729] T 00000000000000000000000000000000 P 272e859e780c4704aaea4c10ad30231f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 272e859e780c4704aaea4c10ad30231f, State: Initialized, Role: FOLLOWER
I20260812 03:12:30.975895 17201 consensus_queue.cc:260] T 00000000000000000000000000000000 P 272e859e780c4704aaea4c10ad30231f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "272e859e780c4704aaea4c10ad30231f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35469 } }
I20260812 03:12:30.976081 17201 raft_consensus.cc:388] T 00000000000000000000000000000000 P 272e859e780c4704aaea4c10ad30231f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:12:30.976181 17201 raft_consensus.cc:482] T 00000000000000000000000000000000 P 272e859e780c4704aaea4c10ad30231f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:12:30.976298 17201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 272e859e780c4704aaea4c10ad30231f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:30.976907 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:30.972745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36533 (local address 127.0.0.1:35469)
0812 03:12:30.972892 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:30.972898 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:30.973088 (+ 190us) server_negotiation.cc:407] Connection header received
0812 03:12:30.973223 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:30.973226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:30.973272 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:30.973358 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:30.973625 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:30.974163 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:30.975040 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:30.975289 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:30.975418 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:30.976477 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:30.976481 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:30.976483 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:30.976496 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:30.976529 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:30.976540 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:30.976622 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:30.976709 (+ 87us) server_negotiation.cc:299] Negotiation successful
0812 03:12:30.976763 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 03:12:30.977483 17201 raft_consensus.cc:504] T 00000000000000000000000000000000 P 272e859e780c4704aaea4c10ad30231f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "272e859e780c4704aaea4c10ad30231f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35469 } }
I20260812 03:12:30.977872 17201 leader_election.cc:302] T 00000000000000000000000000000000 P 272e859e780c4704aaea4c10ad30231f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 272e859e780c4704aaea4c10ad30231f; no voters:
I20260812 03:12:30.978230 17201 leader_election.cc:288] T 00000000000000000000000000000000 P 272e859e780c4704aaea4c10ad30231f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:12:30.978310 17211 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 272e859e780c4704aaea4c10ad30231f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:30.978662 17211 raft_consensus.cc:686] T 00000000000000000000000000000000 P 272e859e780c4704aaea4c10ad30231f [term 1 LEADER]: Becoming Leader. State: Replica: 272e859e780c4704aaea4c10ad30231f, State: Running, Role: LEADER
I20260812 03:12:30.979000 17211 consensus_queue.cc:237] T 00000000000000000000000000000000 P 272e859e780c4704aaea4c10ad30231f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "272e859e780c4704aaea4c10ad30231f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35469 } }
I20260812 03:12:30.979655 17201 sys_catalog.cc:564] T 00000000000000000000000000000000 P 272e859e780c4704aaea4c10ad30231f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:30.980332 17211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 272e859e780c4704aaea4c10ad30231f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 272e859e780c4704aaea4c10ad30231f. Latest consensus state: current_term: 1 leader_uuid: "272e859e780c4704aaea4c10ad30231f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "272e859e780c4704aaea4c10ad30231f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35469 } } }
I20260812 03:12:30.980455 17211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 272e859e780c4704aaea4c10ad30231f [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:30.980872 17214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 272e859e780c4704aaea4c10ad30231f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "272e859e780c4704aaea4c10ad30231f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "272e859e780c4704aaea4c10ad30231f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35469 } } }
I20260812 03:12:30.980994 17214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 272e859e780c4704aaea4c10ad30231f [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:30.982373 17217 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:12:30.985519 17217 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:12:30.992005 17217 catalog_manager.cc:1347] Generated new cluster ID: 535e943766a64323a0b4339946a0ce1d
I20260812 03:12:30.992094 17217 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:12:31.021390 17217 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:12:31.022742 17217 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:12:31.035718 17217 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 272e859e780c4704aaea4c10ad30231f: Generated new TSK 0
I20260812 03:12:31.036540 17217 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:12:31.048557 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 17126
2026-08-12T03:12:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1543 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:12:31 dist-test-slave-zpfg krb5kdc[17236](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:12:31 dist-test-slave-zpfg krb5kdc[17236](info): set up 2 sockets
Aug 12 03:12:31 dist-test-slave-zpfg krb5kdc[17236](info): commencing operation
krb5kdc: starting...
W20260812 03:12:32.152938 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:12:32 dist-test-slave-zpfg krb5kdc[17236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:12:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:12:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:12:32.299394 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35085
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:32.421653 17250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:32.422005 17250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:32.422111 17250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:32.426601 17250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:32.426713 17250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:32.426748 17250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:32.426792 17250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:32.426829 17250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:32.432859 17250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:32.434129 17250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:32.435362 17250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:32.444545 17258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:32.444546 17256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:32.444550 17255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:32.444960 17250 server_base.cc:1034] running on GCE node
I20260812 03:12:32.445531 17250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:32.446821 17250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:32.448038 17250 hybrid_clock.cc:648] HybridClock initialized: now 1786504352448021 us; error 49 us; skew 500 ppm
Aug 12 03:12:32 dist-test-slave-zpfg krb5kdc[17236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504352, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:32.455554 17250 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:12:32.456691 17250 webserver.cc:460] Webserver started at http://127.0.0.1:44455/ using document root <none> and password file <none>
I20260812 03:12:32.457325 17250 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:32.457438 17250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:32.457693 17250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:32.460042 17250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "4ce04768ce4446ffad0f5071db048172"
format_stamp: "Formatted at 2026-08-12 03:12:32 on dist-test-slave-zpfg"
server_key: "8face1b7de198bc8d5439399ffc94014"
server_key_iv: "541831f971c602a6d08abe5db1c593c6"
server_key_version: "encryptionkey@0"
I20260812 03:12:32.460726 17250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "4ce04768ce4446ffad0f5071db048172"
format_stamp: "Formatted at 2026-08-12 03:12:32 on dist-test-slave-zpfg"
server_key: "8face1b7de198bc8d5439399ffc94014"
server_key_iv: "541831f971c602a6d08abe5db1c593c6"
server_key_version: "encryptionkey@0"
I20260812 03:12:32.464942 17250 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:12:32.467792 17265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:32.469136 17250 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:12:32.469306 17250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "4ce04768ce4446ffad0f5071db048172"
format_stamp: "Formatted at 2026-08-12 03:12:32 on dist-test-slave-zpfg"
server_key: "8face1b7de198bc8d5439399ffc94014"
server_key_iv: "541831f971c602a6d08abe5db1c593c6"
server_key_version: "encryptionkey@0"
I20260812 03:12:32.469444 17250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:32.474359 17250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:32.477562 17250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:32.477773 17250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:32.485693 17250 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35085
I20260812 03:12:32.485728 17322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35085 every 8 connection(s)
I20260812 03:12:32.486737 17250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:12:32.487718 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 17250
I20260812 03:12:32.487834 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:12:32.488128 27626 external_mini_cluster.cc:1428] Setting key a586cb9df433a1e2ff69b9b3d5e36a3e
I20260812 03:12:32.490253 17323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:32.494244 17326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:32.489779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55903 (local address 127.0.0.1:35085)
0812 03:12:32.490231 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:32.490253 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:32.490300 (+ 47us) server_negotiation.cc:407] Connection header received
0812 03:12:32.491207 (+ 907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:32.491230 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:32.491588 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:32.491971 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:32.492736 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:32.492744 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:32.492760 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:32.492772 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:32.492837 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:32.492866 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:32.493016 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:32.493088 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 03:12:32.493261 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":123,"threads_started":1}
I20260812 03:12:32.497190 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:12:32.497421 17323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ce04768ce4446ffad0f5071db048172: Bootstrap starting.
I20260812 03:12:32.500099 17323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ce04768ce4446ffad0f5071db048172: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:32.501175 17323 log.cc:824] T 00000000000000000000000000000000 P 4ce04768ce4446ffad0f5071db048172: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:32.501367 17326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:32.499536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57965 (local address 127.0.0.1:35085)
0812 03:12:32.499682 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:32.499687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:32.499912 (+ 225us) server_negotiation.cc:407] Connection header received
0812 03:12:32.500061 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:32.500063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:32.500096 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:32.500164 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:32.500931 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:32.500934 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:32.500936 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:32.500946 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:32.500972 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:32.500983 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:32.501055 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:32.501187 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 03:12:32.501223 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 03:12:32.503626 17323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ce04768ce4446ffad0f5071db048172: No bootstrap required, opened a new log
I20260812 03:12:32.508131 17323 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ce04768ce4446ffad0f5071db048172 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce04768ce4446ffad0f5071db048172" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35085 } }
I20260812 03:12:32.508404 17323 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ce04768ce4446ffad0f5071db048172 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:32.508473 17323 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ce04768ce4446ffad0f5071db048172 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ce04768ce4446ffad0f5071db048172, State: Initialized, Role: FOLLOWER
I20260812 03:12:32.508987 17323 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ce04768ce4446ffad0f5071db048172 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce04768ce4446ffad0f5071db048172" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35085 } }
I20260812 03:12:32.509123 17323 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ce04768ce4446ffad0f5071db048172 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:12:32.509224 17323 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ce04768ce4446ffad0f5071db048172 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:12:32.509373 17323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ce04768ce4446ffad0f5071db048172 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:32.510543 17323 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ce04768ce4446ffad0f5071db048172 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce04768ce4446ffad0f5071db048172" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35085 } }
I20260812 03:12:32.510942 17323 leader_election.cc:302] T 00000000000000000000000000000000 P 4ce04768ce4446ffad0f5071db048172 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ce04768ce4446ffad0f5071db048172; no voters:
I20260812 03:12:32.511353 17323 leader_election.cc:288] T 00000000000000000000000000000000 P 4ce04768ce4446ffad0f5071db048172 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:12:32.511492 17333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ce04768ce4446ffad0f5071db048172 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:32.511775 17333 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ce04768ce4446ffad0f5071db048172 [term 1 LEADER]: Becoming Leader. State: Replica: 4ce04768ce4446ffad0f5071db048172, State: Running, Role: LEADER
I20260812 03:12:32.512195 17333 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ce04768ce4446ffad0f5071db048172 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce04768ce4446ffad0f5071db048172" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35085 } }
I20260812 03:12:32.512813 17323 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ce04768ce4446ffad0f5071db048172 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:32.514020 17334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ce04768ce4446ffad0f5071db048172 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ce04768ce4446ffad0f5071db048172" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce04768ce4446ffad0f5071db048172" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35085 } } }
I20260812 03:12:32.514158 17334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ce04768ce4446ffad0f5071db048172 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:32.514523 17335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ce04768ce4446ffad0f5071db048172 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ce04768ce4446ffad0f5071db048172. Latest consensus state: current_term: 1 leader_uuid: "4ce04768ce4446ffad0f5071db048172" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ce04768ce4446ffad0f5071db048172" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35085 } } }
I20260812 03:12:32.514602 17342 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:12:32.514618 17335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ce04768ce4446ffad0f5071db048172 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:32.517609 17342 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:12:32.524149 17342 catalog_manager.cc:1347] Generated new cluster ID: 1c82c2f06af445c781260dab4af0c88e
I20260812 03:12:32.524236 17342 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:12:32.537313 17342 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:12:32.537693 17342 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:12:32.545506 17342 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ce04768ce4446ffad0f5071db048172: Generated new TSK 0
I20260812 03:12:32.546268 17342 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:12:32.556398 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 17250
2026-08-12T03:12:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1508 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:12:32 dist-test-slave-zpfg krb5kdc[17354](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:12:32 dist-test-slave-zpfg krb5kdc[17354](info): set up 2 sockets
Aug 12 03:12:32 dist-test-slave-zpfg krb5kdc[17354](info): commencing operation
krb5kdc: starting...
W20260812 03:12:33.675621 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:12:33 dist-test-slave-zpfg krb5kdc[17354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:12:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:12:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:12:33.826654 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36285
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36285
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:33.950111 17370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:33.950502 17370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:33.950587 17370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:33.954809 17370 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:33.954883 17370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:33.954903 17370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:33.954921 17370 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:33.954936 17370 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:33.960693 17370 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36285
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:33.961926 17370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:33.963127 17370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:33.972394 17379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:33.972417 17376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:33.972496 17370 server_base.cc:1034] running on GCE node
W20260812 03:12:33.972394 17377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:33.973354 17370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:33.974684 17370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:33.975894 17370 hybrid_clock.cc:648] HybridClock initialized: now 1786504353975866 us; error 53 us; skew 500 ppm
Aug 12 03:12:33 dist-test-slave-zpfg krb5kdc[17354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504353, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:33.978907 17370 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:12:33.979861 17370 webserver.cc:460] Webserver started at http://127.0.0.1:38697/ using document root <none> and password file <none>
I20260812 03:12:33.980428 17370 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:33.980490 17370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:33.980661 17370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:33.982846 17370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "7fd8504921e04cd188929c205e48fd80"
format_stamp: "Formatted at 2026-08-12 03:12:33 on dist-test-slave-zpfg"
server_key: "ecb6b7b3725daf80a7b4aa99e60bb843"
server_key_iv: "d0693528b074e4f93671aacccb74be69"
server_key_version: "encryptionkey@0"
I20260812 03:12:33.983397 17370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "7fd8504921e04cd188929c205e48fd80"
format_stamp: "Formatted at 2026-08-12 03:12:33 on dist-test-slave-zpfg"
server_key: "ecb6b7b3725daf80a7b4aa99e60bb843"
server_key_iv: "d0693528b074e4f93671aacccb74be69"
server_key_version: "encryptionkey@0"
I20260812 03:12:33.987397 17370 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 03:12:33.989998 17388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:33.991433 17370 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 03:12:33.991587 17370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "7fd8504921e04cd188929c205e48fd80"
format_stamp: "Formatted at 2026-08-12 03:12:33 on dist-test-slave-zpfg"
server_key: "ecb6b7b3725daf80a7b4aa99e60bb843"
server_key_iv: "d0693528b074e4f93671aacccb74be69"
server_key_version: "encryptionkey@0"
I20260812 03:12:33.991701 17370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:34.032522 17370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:34.036685 17370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:34.036947 17370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:34.044781 17370 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36285
I20260812 03:12:34.044804 17444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36285 every 8 connection(s)
I20260812 03:12:34.045874 17370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:12:34.048856 17445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:34.054629 17445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fd8504921e04cd188929c205e48fd80: Bootstrap starting.
I20260812 03:12:34.055963 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 17370
I20260812 03:12:34.056106 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:12:34.056465 27626 external_mini_cluster.cc:1428] Setting key c69c9d99587785aa8d9e80b3cc219269
I20260812 03:12:34.057492 17445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7fd8504921e04cd188929c205e48fd80: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:34.058411 17445 log.cc:824] T 00000000000000000000000000000000 P 7fd8504921e04cd188929c205e48fd80: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:34.060529 17445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fd8504921e04cd188929c205e48fd80: No bootstrap required, opened a new log
I20260812 03:12:34.064476 17445 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7fd8504921e04cd188929c205e48fd80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fd8504921e04cd188929c205e48fd80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36285 } }
I20260812 03:12:34.064785 17445 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7fd8504921e04cd188929c205e48fd80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:34.064883 17445 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7fd8504921e04cd188929c205e48fd80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fd8504921e04cd188929c205e48fd80, State: Initialized, Role: FOLLOWER
I20260812 03:12:34.065364 17445 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7fd8504921e04cd188929c205e48fd80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fd8504921e04cd188929c205e48fd80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36285 } }
I20260812 03:12:34.065526 17445 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7fd8504921e04cd188929c205e48fd80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:12:34.065618 17445 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7fd8504921e04cd188929c205e48fd80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:12:34.065754 17445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7fd8504921e04cd188929c205e48fd80 [term 0 FOLLOWER]: Advancing to term 1
Aug 12 03:12:34 dist-test-slave-zpfg krb5kdc[17354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 03:12:34.066929 17445 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7fd8504921e04cd188929c205e48fd80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fd8504921e04cd188929c205e48fd80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36285 } }
I20260812 03:12:34.067332 17445 leader_election.cc:302] T 00000000000000000000000000000000 P 7fd8504921e04cd188929c205e48fd80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7fd8504921e04cd188929c205e48fd80; no voters:
I20260812 03:12:34.067785 17445 leader_election.cc:288] T 00000000000000000000000000000000 P 7fd8504921e04cd188929c205e48fd80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:12:34.067869 17450 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7fd8504921e04cd188929c205e48fd80 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:34.068491 17450 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7fd8504921e04cd188929c205e48fd80 [term 1 LEADER]: Becoming Leader. State: Replica: 7fd8504921e04cd188929c205e48fd80, State: Running, Role: LEADER
I20260812 03:12:34.068825 17450 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7fd8504921e04cd188929c205e48fd80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fd8504921e04cd188929c205e48fd80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36285 } }
I20260812 03:12:34.069325 17445 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7fd8504921e04cd188929c205e48fd80 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:34.070634 17452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fd8504921e04cd188929c205e48fd80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7fd8504921e04cd188929c205e48fd80. Latest consensus state: current_term: 1 leader_uuid: "7fd8504921e04cd188929c205e48fd80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fd8504921e04cd188929c205e48fd80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36285 } } }
I20260812 03:12:34.070772 17452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fd8504921e04cd188929c205e48fd80 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:34.071133 17451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fd8504921e04cd188929c205e48fd80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7fd8504921e04cd188929c205e48fd80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fd8504921e04cd188929c205e48fd80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36285 } } }
I20260812 03:12:34.071250 17451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fd8504921e04cd188929c205e48fd80 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:34.073737 17448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:34.057735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:56703 (local address 127.0.0.1:36285)
0812 03:12:34.058255 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:34.058275 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:34.058325 (+ 50us) server_negotiation.cc:407] Connection header received
0812 03:12:34.059369 (+ 1044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:34.059389 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:34.059755 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:34.060135 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:34.061028 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:34.062031 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:34.063101 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:34.063617 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:34.063807 (+ 190us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:34.066927 (+ 3120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:34.066932 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:34.066948 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:34.066980 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:34.069181 (+ 2201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:34.069865 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:34.069870 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:34.069875 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:34.070053 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:34.070573 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:34.070578 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:34.070581 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:34.071186 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:34.072232 (+ 1046us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:34.072436 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 03:12:34.072673 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":155,"threads_started":1}
W20260812 03:12:34.074457 17465 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7fd8504921e04cd188929c205e48fd80: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:12:34.074539 17465 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:12:34.074602 17460 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:12:34.075477 17460 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:12:34.076897 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:12:34.082484 17460 catalog_manager.cc:1347] Generated new cluster ID: d2c15d61b0cc473b90ea421dc68cacd3
I20260812 03:12:34.082568 17460 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:12:34.089159 17448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:34.079233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:54227 (local address 127.0.0.1:36285)
0812 03:12:34.079489 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:34.079494 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:34.079725 (+ 231us) server_negotiation.cc:407] Connection header received
0812 03:12:34.079886 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:34.079891 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:34.079983 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:34.080080 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:34.080954 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:34.081786 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:34.083032 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:34.083379 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:34.083494 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:34.085241 (+ 1747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:34.085245 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:34.085247 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:34.085274 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:34.087005 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:34.087575 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:34.087578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:34.087580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:34.087667 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:34.088058 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:34.088061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:34.088063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:34.088270 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:34.088400 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:34.088857 (+ 457us) server_negotiation.cc:299] Negotiation successful
0812 03:12:34.088991 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171}
I20260812 03:12:34.089689 17460 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:12:34.090768 17460 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:12:34.096189 17460 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7fd8504921e04cd188929c205e48fd80: Generated new TSK 0
I20260812 03:12:34.097168 17460 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:12:34.102751 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 17370
W20260812 03:12:34.111424 17352 connection.cc:535] client connection to 127.0.0.1:36285 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36285): Connection reset by peer (error 104)
2026-08-12T03:12:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1545 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:12:34 dist-test-slave-zpfg krb5kdc[17477](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:12:34 dist-test-slave-zpfg krb5kdc[17477](info): set up 2 sockets
Aug 12 03:12:34 dist-test-slave-zpfg krb5kdc[17477](info): commencing operation
krb5kdc: starting...
W20260812 03:12:35.222242 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:12:35 dist-test-slave-zpfg krb5kdc[17477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:12:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:12:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:12:35.363619 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37749
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:35.485905 17491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:35.486235 17491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:35.486280 17491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:35.490307 17491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:35.490406 17491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:35.490427 17491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:35.490444 17491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:35.490459 17491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:35.496315 17491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:35.497565 17491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:35.498952 17491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:35.508272 17502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:35.508394 17499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:35.508286 17498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:35.508582 17491 server_base.cc:1034] running on GCE node
I20260812 03:12:35.509326 17491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:35.510818 17491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:35.512014 17491 hybrid_clock.cc:648] HybridClock initialized: now 1786504355511995 us; error 34 us; skew 500 ppm
Aug 12 03:12:35 dist-test-slave-zpfg krb5kdc[17477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504355, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:35.515380 17491 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:12:35.516386 17491 webserver.cc:460] Webserver started at http://127.0.0.1:33625/ using document root <none> and password file <none>
I20260812 03:12:35.516971 17491 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:35.517037 17491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:35.517222 17491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:35.519554 17491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "e82fec7407ce477fad65a6658e26cb2a"
format_stamp: "Formatted at 2026-08-12 03:12:35 on dist-test-slave-zpfg"
server_key: "e907de04889ef4fc42d85e202e2c5432"
server_key_iv: "155bdd3207909f42a263b0ffddb468e7"
server_key_version: "encryptionkey@0"
I20260812 03:12:35.520138 17491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "e82fec7407ce477fad65a6658e26cb2a"
format_stamp: "Formatted at 2026-08-12 03:12:35 on dist-test-slave-zpfg"
server_key: "e907de04889ef4fc42d85e202e2c5432"
server_key_iv: "155bdd3207909f42a263b0ffddb468e7"
server_key_version: "encryptionkey@0"
I20260812 03:12:35.524288 17491 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 03:12:35.527457 17509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:35.529115 17491 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:12:35.529286 17491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "e82fec7407ce477fad65a6658e26cb2a"
format_stamp: "Formatted at 2026-08-12 03:12:35 on dist-test-slave-zpfg"
server_key: "e907de04889ef4fc42d85e202e2c5432"
server_key_iv: "155bdd3207909f42a263b0ffddb468e7"
server_key_version: "encryptionkey@0"
I20260812 03:12:35.529431 17491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:35.551594 17491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:35.555140 17491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:35.555402 17491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:35.563637 17491 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37749
I20260812 03:12:35.563644 17568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37749 every 8 connection(s)
I20260812 03:12:35.564786 17491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:12:35.568033 17569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:35.572932 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 17491
I20260812 03:12:35.573127 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:12:35.573451 27626 external_mini_cluster.cc:1428] Setting key c32df42ea2b4ded668f2740a04067e18
I20260812 03:12:35.574776 17569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e82fec7407ce477fad65a6658e26cb2a: Bootstrap starting.
I20260812 03:12:35.577322 17569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e82fec7407ce477fad65a6658e26cb2a: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:35.578114 17569 log.cc:824] T 00000000000000000000000000000000 P e82fec7407ce477fad65a6658e26cb2a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:35.580215 17569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e82fec7407ce477fad65a6658e26cb2a: No bootstrap required, opened a new log
I20260812 03:12:35.583537 17572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:35.575215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:37069 (local address 127.0.0.1:37749)
0812 03:12:35.575718 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:35.575735 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:35.575773 (+ 38us) server_negotiation.cc:407] Connection header received
0812 03:12:35.576573 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:35.576593 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:35.576975 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:35.577343 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:35.577790 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:35.578934 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:35.579875 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:35.580420 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:35.580589 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:35.581633 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:35.581639 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:35.581655 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:35.582081 (+ 426us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:35.582149 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:35.582173 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:35.582324 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:35.582572 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 03:12:35.582736 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":146,"threads_started":1}
I20260812 03:12:35.584311 17569 raft_consensus.cc:348] T 00000000000000000000000000000000 P e82fec7407ce477fad65a6658e26cb2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e82fec7407ce477fad65a6658e26cb2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37749 } }
I20260812 03:12:35.584637 17569 raft_consensus.cc:374] T 00000000000000000000000000000000 P e82fec7407ce477fad65a6658e26cb2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:35.584692 17569 raft_consensus.cc:729] T 00000000000000000000000000000000 P e82fec7407ce477fad65a6658e26cb2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82fec7407ce477fad65a6658e26cb2a, State: Initialized, Role: FOLLOWER
I20260812 03:12:35.585212 17569 consensus_queue.cc:260] T 00000000000000000000000000000000 P e82fec7407ce477fad65a6658e26cb2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e82fec7407ce477fad65a6658e26cb2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37749 } }
I20260812 03:12:35.585336 17569 raft_consensus.cc:388] T 00000000000000000000000000000000 P e82fec7407ce477fad65a6658e26cb2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:12:35.585389 17569 raft_consensus.cc:482] T 00000000000000000000000000000000 P e82fec7407ce477fad65a6658e26cb2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:12:35.585455 17569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e82fec7407ce477fad65a6658e26cb2a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:35.586706 17569 raft_consensus.cc:504] T 00000000000000000000000000000000 P e82fec7407ce477fad65a6658e26cb2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e82fec7407ce477fad65a6658e26cb2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37749 } }
I20260812 03:12:35.587109 17569 leader_election.cc:302] T 00000000000000000000000000000000 P e82fec7407ce477fad65a6658e26cb2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e82fec7407ce477fad65a6658e26cb2a; no voters:
I20260812 03:12:35.587340 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:12:35.587533 17569 leader_election.cc:288] T 00000000000000000000000000000000 P e82fec7407ce477fad65a6658e26cb2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:12:35.587671 17574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e82fec7407ce477fad65a6658e26cb2a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:35.587977 17574 raft_consensus.cc:686] T 00000000000000000000000000000000 P e82fec7407ce477fad65a6658e26cb2a [term 1 LEADER]: Becoming Leader. State: Replica: e82fec7407ce477fad65a6658e26cb2a, State: Running, Role: LEADER
I20260812 03:12:35.588403 17574 consensus_queue.cc:237] T 00000000000000000000000000000000 P e82fec7407ce477fad65a6658e26cb2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e82fec7407ce477fad65a6658e26cb2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37749 } }
I20260812 03:12:35.588982 17569 sys_catalog.cc:564] T 00000000000000000000000000000000 P e82fec7407ce477fad65a6658e26cb2a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:35.589735 17575 sys_catalog.cc:455] T 00000000000000000000000000000000 P e82fec7407ce477fad65a6658e26cb2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e82fec7407ce477fad65a6658e26cb2a. Latest consensus state: current_term: 1 leader_uuid: "e82fec7407ce477fad65a6658e26cb2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e82fec7407ce477fad65a6658e26cb2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37749 } } }
I20260812 03:12:35.589881 17575 sys_catalog.cc:458] T 00000000000000000000000000000000 P e82fec7407ce477fad65a6658e26cb2a [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:35.590269 17576 sys_catalog.cc:455] T 00000000000000000000000000000000 P e82fec7407ce477fad65a6658e26cb2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e82fec7407ce477fad65a6658e26cb2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e82fec7407ce477fad65a6658e26cb2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37749 } } }
I20260812 03:12:35.590425 17576 sys_catalog.cc:458] T 00000000000000000000000000000000 P e82fec7407ce477fad65a6658e26cb2a [sys.catalog]: This master's current role is: LEADER
W20260812 03:12:35.594755 17597 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e82fec7407ce477fad65a6658e26cb2a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:12:35.594852 17597 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:12:35.594936 17586 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:12:35.595801 17586 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:12:35.595844 17572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:35.590785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:57409 (local address 127.0.0.1:37749)
0812 03:12:35.590940 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:35.590951 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:35.591727 (+ 776us) server_negotiation.cc:407] Connection header received
0812 03:12:35.591929 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:35.591933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:35.591986 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:35.592078 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:35.592549 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:35.593079 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:35.593943 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:35.594183 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:35.594296 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:35.595282 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:35.595285 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:35.595290 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:35.595308 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:35.595333 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:35.595344 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:35.595446 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:35.595601 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 03:12:35.595637 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 03:12:35.601719 17586 catalog_manager.cc:1347] Generated new cluster ID: 47f64cf8985a441ca630a67fddb42c79
I20260812 03:12:35.601802 17586 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:12:35.636651 17586 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:12:35.637741 17586 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:12:35.641582 17586 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e82fec7407ce477fad65a6658e26cb2a: Generated new TSK 0
I20260812 03:12:35.642364 17586 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:12:35.648960 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 17491
W20260812 03:12:35.658680 17475 connection.cc:535] client connection to 127.0.0.1:37749 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37749): Connection reset by peer (error 104)
2026-08-12T03:12:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1547 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:12:35 dist-test-slave-zpfg krb5kdc[17604](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:12:35 dist-test-slave-zpfg krb5kdc[17604](info): set up 2 sockets
Aug 12 03:12:35 dist-test-slave-zpfg krb5kdc[17604](info): commencing operation
krb5kdc: starting...
W20260812 03:12:36.835043 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:12:36 dist-test-slave-zpfg krb5kdc[17604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:12:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:12:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:12:36.999045 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41243
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:37.133409 17617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:37.133932 17617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:37.134045 17617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:37.138904 17617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:37.139011 17617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:37.139045 17617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:37.139284 17617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:37.139459 17617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:37.145859 17617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:37.147341 17617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:37.148533 17617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:37.159461 17625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:37.159440 17622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:37.159487 17623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:37.159758 17617 server_base.cc:1034] running on GCE node
I20260812 03:12:37.160594 17617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:37.161949 17617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:37.163178 17617 hybrid_clock.cc:648] HybridClock initialized: now 1786504357163180 us; error 100 us; skew 500 ppm
Aug 12 03:12:37 dist-test-slave-zpfg krb5kdc[17604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504357, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:37.167032 17617 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:12:37.169209 17617 webserver.cc:460] Webserver started at http://127.0.0.1:43885/ using document root <none> and password file <none>
I20260812 03:12:37.169865 17617 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:37.169970 17617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:37.170234 17617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:37.172734 17617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "28f7242da4c54a8c900c7ed8276e0b45"
format_stamp: "Formatted at 2026-08-12 03:12:37 on dist-test-slave-zpfg"
server_key: "f23d43c40c948a76b4c0ce4fc76fc227"
server_key_iv: "63dab94bcbad25e065dbb3991df35149"
server_key_version: "encryptionkey@0"
I20260812 03:12:37.173453 17617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "28f7242da4c54a8c900c7ed8276e0b45"
format_stamp: "Formatted at 2026-08-12 03:12:37 on dist-test-slave-zpfg"
server_key: "f23d43c40c948a76b4c0ce4fc76fc227"
server_key_iv: "63dab94bcbad25e065dbb3991df35149"
server_key_version: "encryptionkey@0"
I20260812 03:12:37.178500 17617 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 03:12:37.182163 17632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:37.183754 17617 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:12:37.183949 17617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "28f7242da4c54a8c900c7ed8276e0b45"
format_stamp: "Formatted at 2026-08-12 03:12:37 on dist-test-slave-zpfg"
server_key: "f23d43c40c948a76b4c0ce4fc76fc227"
server_key_iv: "63dab94bcbad25e065dbb3991df35149"
server_key_version: "encryptionkey@0"
I20260812 03:12:37.184131 17617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:37.190243 17617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:37.193929 17617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:37.194222 17617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:37.202991 17617 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41243
I20260812 03:12:37.203017 17688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41243 every 8 connection(s)
I20260812 03:12:37.204030 17617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:12:37.207739 17689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:37.208775 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 17617
I20260812 03:12:37.209025 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:12:37.209597 27626 external_mini_cluster.cc:1428] Setting key d81769ee26bea05c9eeae465ed45e80d
I20260812 03:12:37.216301 17692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:37.211328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:40785 (local address 127.0.0.1:41243)
0812 03:12:37.211848 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:37.211886 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:37.211946 (+ 60us) server_negotiation.cc:407] Connection header received
0812 03:12:37.212907 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:37.212933 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:37.213324 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:37.213781 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:37.214593 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:37.214599 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:37.214615 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:37.214627 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:37.214692 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:37.214735 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:37.214898 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:37.214972 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 03:12:37.215150 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":156,"threads_started":1}
I20260812 03:12:37.216326 17689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28f7242da4c54a8c900c7ed8276e0b45: Bootstrap starting.
I20260812 03:12:37.219728 17689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28f7242da4c54a8c900c7ed8276e0b45: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:37.220722 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:12:37.220845 17689 log.cc:824] T 00000000000000000000000000000000 P 28f7242da4c54a8c900c7ed8276e0b45: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:37.223798 17689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28f7242da4c54a8c900c7ed8276e0b45: No bootstrap required, opened a new log
I20260812 03:12:37.225894 17692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:37.223465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:47499 (local address 127.0.0.1:41243)
0812 03:12:37.223621 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:37.223626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:37.223912 (+ 286us) server_negotiation.cc:407] Connection header received
0812 03:12:37.224028 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:37.224031 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:37.224065 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:37.224136 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:37.225261 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:37.225264 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:37.225266 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:37.225277 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:37.225303 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:37.225314 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:37.225404 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:37.225554 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 03:12:37.225724 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 03:12:37.229430 17689 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28f7242da4c54a8c900c7ed8276e0b45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28f7242da4c54a8c900c7ed8276e0b45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41243 } }
I20260812 03:12:37.229780 17689 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28f7242da4c54a8c900c7ed8276e0b45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:37.229830 17689 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28f7242da4c54a8c900c7ed8276e0b45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28f7242da4c54a8c900c7ed8276e0b45, State: Initialized, Role: FOLLOWER
I20260812 03:12:37.230378 17689 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28f7242da4c54a8c900c7ed8276e0b45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28f7242da4c54a8c900c7ed8276e0b45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41243 } }
I20260812 03:12:37.230551 17689 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28f7242da4c54a8c900c7ed8276e0b45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:12:37.230655 17689 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28f7242da4c54a8c900c7ed8276e0b45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:12:37.230803 17689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28f7242da4c54a8c900c7ed8276e0b45 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:37.232558 17689 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28f7242da4c54a8c900c7ed8276e0b45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28f7242da4c54a8c900c7ed8276e0b45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41243 } }
I20260812 03:12:37.233047 17689 leader_election.cc:302] T 00000000000000000000000000000000 P 28f7242da4c54a8c900c7ed8276e0b45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28f7242da4c54a8c900c7ed8276e0b45; no voters:
I20260812 03:12:37.233566 17689 leader_election.cc:288] T 00000000000000000000000000000000 P 28f7242da4c54a8c900c7ed8276e0b45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:12:37.233712 17700 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28f7242da4c54a8c900c7ed8276e0b45 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:37.233985 17700 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28f7242da4c54a8c900c7ed8276e0b45 [term 1 LEADER]: Becoming Leader. State: Replica: 28f7242da4c54a8c900c7ed8276e0b45, State: Running, Role: LEADER
I20260812 03:12:37.234465 17700 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28f7242da4c54a8c900c7ed8276e0b45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28f7242da4c54a8c900c7ed8276e0b45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41243 } }
I20260812 03:12:37.235222 17689 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28f7242da4c54a8c900c7ed8276e0b45 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:37.236768 17702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28f7242da4c54a8c900c7ed8276e0b45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28f7242da4c54a8c900c7ed8276e0b45. Latest consensus state: current_term: 1 leader_uuid: "28f7242da4c54a8c900c7ed8276e0b45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28f7242da4c54a8c900c7ed8276e0b45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41243 } } }
I20260812 03:12:37.236931 17702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28f7242da4c54a8c900c7ed8276e0b45 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:37.236766 17701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28f7242da4c54a8c900c7ed8276e0b45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28f7242da4c54a8c900c7ed8276e0b45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28f7242da4c54a8c900c7ed8276e0b45" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41243 } } }
I20260812 03:12:37.237146 17701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28f7242da4c54a8c900c7ed8276e0b45 [sys.catalog]: This master's current role is: LEADER
W20260812 03:12:37.241782 17715 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 28f7242da4c54a8c900c7ed8276e0b45: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:12:37.241907 17715 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:12:37.242100 17709 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:12:37.243491 17709 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:12:37.250721 17709 catalog_manager.cc:1347] Generated new cluster ID: b39bbfda2f314f41a4cc5c5e9cad521e
I20260812 03:12:37.250882 17709 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:12:37.269516 17709 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:12:37.269987 17709 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:12:37.293632 17709 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28f7242da4c54a8c900c7ed8276e0b45: Generated new TSK 0
I20260812 03:12:37.294762 17709 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:12:37.311637 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 17617
2026-08-12T03:12:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1664 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:12:37 dist-test-slave-zpfg krb5kdc[17722](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:12:37 dist-test-slave-zpfg krb5kdc[17722](info): set up 2 sockets
Aug 12 03:12:37 dist-test-slave-zpfg krb5kdc[17722](info): commencing operation
krb5kdc: starting...
W20260812 03:12:38.547271 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:12:38 dist-test-slave-zpfg krb5kdc[17722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:12:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:12:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:12:38.694908 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38231
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:38.819681 17741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:38.820036 17741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:38.820128 17741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:38.824549 17741 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:38.824678 17741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:38.824718 17741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:38.824762 17741 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:38.824801 17741 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:38.830662 17741 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38231
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:38.831990 17741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:38.833199 17741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 03:12:38.843183 17741 server_base.cc:1034] running on GCE node
W20260812 03:12:38.843129 17749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:38.843145 17747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:38.843129 17746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:38.844058 17741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:38.845386 17741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:38.846601 17741 hybrid_clock.cc:648] HybridClock initialized: now 1786504358846563 us; error 51 us; skew 500 ppm
Aug 12 03:12:38 dist-test-slave-zpfg krb5kdc[17722](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504358, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:38.857396 17741 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:12:38.858659 17741 webserver.cc:460] Webserver started at http://127.0.0.1:35137/ using document root <none> and password file <none>
I20260812 03:12:38.859313 17741 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:38.859414 17741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:38.859690 17741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:38.862001 17741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "e77b251a801a411aaacc16ee7382bbe7"
format_stamp: "Formatted at 2026-08-12 03:12:38 on dist-test-slave-zpfg"
server_key: "82e926a818b6db9a90d6bd2e1d573001"
server_key_iv: "7a77a994310f72a5405848c4cc303124"
server_key_version: "encryptionkey@0"
I20260812 03:12:38.862694 17741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "e77b251a801a411aaacc16ee7382bbe7"
format_stamp: "Formatted at 2026-08-12 03:12:38 on dist-test-slave-zpfg"
server_key: "82e926a818b6db9a90d6bd2e1d573001"
server_key_iv: "7a77a994310f72a5405848c4cc303124"
server_key_version: "encryptionkey@0"
I20260812 03:12:38.867261 17741 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 03:12:38.870275 17757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:38.872016 17741 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 03:12:38.872211 17741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "e77b251a801a411aaacc16ee7382bbe7"
format_stamp: "Formatted at 2026-08-12 03:12:38 on dist-test-slave-zpfg"
server_key: "82e926a818b6db9a90d6bd2e1d573001"
server_key_iv: "7a77a994310f72a5405848c4cc303124"
server_key_version: "encryptionkey@0"
I20260812 03:12:38.872383 17741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:38.895277 17741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:38.899137 17741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:38.899374 17741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:38.908245 17741 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38231
I20260812 03:12:38.908257 17811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38231 every 8 connection(s)
I20260812 03:12:38.909452 17741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:12:38.912724 17812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:38.913664 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 17741
I20260812 03:12:38.913820 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:12:38.914175 27626 external_mini_cluster.cc:1428] Setting key a8c30c82329cf1b0bafc9704377d1a2b
I20260812 03:12:38.919905 17812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e77b251a801a411aaacc16ee7382bbe7: Bootstrap starting.
I20260812 03:12:38.922585 17812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e77b251a801a411aaacc16ee7382bbe7: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:38.923475 17812 log.cc:824] T 00000000000000000000000000000000 P e77b251a801a411aaacc16ee7382bbe7: Log is configured to *not* fsync() on all Append() calls
Aug 12 03:12:38 dist-test-slave-zpfg krb5kdc[17722](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 03:12:38.926280 17812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e77b251a801a411aaacc16ee7382bbe7: No bootstrap required, opened a new log
I20260812 03:12:38.931176 17815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:38.915957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:35977 (local address 127.0.0.1:38231)
0812 03:12:38.916561 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:38.916577 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:38.916619 (+ 42us) server_negotiation.cc:407] Connection header received
0812 03:12:38.917285 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:38.917304 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:38.917718 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:38.918169 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:38.919118 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:38.920238 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:38.921169 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:38.921695 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:38.921942 (+ 247us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:38.925114 (+ 3172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:38.925120 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:38.925136 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:38.925164 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:38.927314 (+ 2150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:38.928031 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:38.928036 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:38.928041 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:38.928176 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:38.928614 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:38.928616 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:38.928618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:38.929079 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:38.929264 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:38.929865 (+ 601us) server_negotiation.cc:299] Negotiation successful
0812 03:12:38.930193 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":189,"threads_started":1}
I20260812 03:12:38.931727 17812 raft_consensus.cc:348] T 00000000000000000000000000000000 P e77b251a801a411aaacc16ee7382bbe7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e77b251a801a411aaacc16ee7382bbe7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38231 } }
I20260812 03:12:38.932040 17812 raft_consensus.cc:374] T 00000000000000000000000000000000 P e77b251a801a411aaacc16ee7382bbe7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:38.932176 17812 raft_consensus.cc:729] T 00000000000000000000000000000000 P e77b251a801a411aaacc16ee7382bbe7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e77b251a801a411aaacc16ee7382bbe7, State: Initialized, Role: FOLLOWER
I20260812 03:12:38.932780 17812 consensus_queue.cc:260] T 00000000000000000000000000000000 P e77b251a801a411aaacc16ee7382bbe7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e77b251a801a411aaacc16ee7382bbe7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38231 } }
I20260812 03:12:38.932976 17812 raft_consensus.cc:388] T 00000000000000000000000000000000 P e77b251a801a411aaacc16ee7382bbe7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:12:38.933043 17812 raft_consensus.cc:482] T 00000000000000000000000000000000 P e77b251a801a411aaacc16ee7382bbe7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:12:38.933166 17812 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e77b251a801a411aaacc16ee7382bbe7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:38.934677 17812 raft_consensus.cc:504] T 00000000000000000000000000000000 P e77b251a801a411aaacc16ee7382bbe7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e77b251a801a411aaacc16ee7382bbe7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38231 } }
I20260812 03:12:38.935164 17812 leader_election.cc:302] T 00000000000000000000000000000000 P e77b251a801a411aaacc16ee7382bbe7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e77b251a801a411aaacc16ee7382bbe7; no voters:
I20260812 03:12:38.935405 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:12:38.935660 17812 leader_election.cc:288] T 00000000000000000000000000000000 P e77b251a801a411aaacc16ee7382bbe7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:12:38.935839 17817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e77b251a801a411aaacc16ee7382bbe7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:38.936118 17817 raft_consensus.cc:686] T 00000000000000000000000000000000 P e77b251a801a411aaacc16ee7382bbe7 [term 1 LEADER]: Becoming Leader. State: Replica: e77b251a801a411aaacc16ee7382bbe7, State: Running, Role: LEADER
I20260812 03:12:38.936610 17817 consensus_queue.cc:237] T 00000000000000000000000000000000 P e77b251a801a411aaacc16ee7382bbe7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e77b251a801a411aaacc16ee7382bbe7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38231 } }
I20260812 03:12:38.937320 17812 sys_catalog.cc:564] T 00000000000000000000000000000000 P e77b251a801a411aaacc16ee7382bbe7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:38.939239 17819 sys_catalog.cc:455] T 00000000000000000000000000000000 P e77b251a801a411aaacc16ee7382bbe7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e77b251a801a411aaacc16ee7382bbe7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e77b251a801a411aaacc16ee7382bbe7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38231 } } }
I20260812 03:12:38.939457 17819 sys_catalog.cc:458] T 00000000000000000000000000000000 P e77b251a801a411aaacc16ee7382bbe7 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:38.939932 17818 sys_catalog.cc:455] T 00000000000000000000000000000000 P e77b251a801a411aaacc16ee7382bbe7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e77b251a801a411aaacc16ee7382bbe7. Latest consensus state: current_term: 1 leader_uuid: "e77b251a801a411aaacc16ee7382bbe7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e77b251a801a411aaacc16ee7382bbe7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38231 } } }
I20260812 03:12:38.940282 17818 sys_catalog.cc:458] T 00000000000000000000000000000000 P e77b251a801a411aaacc16ee7382bbe7 [sys.catalog]: This master's current role is: LEADER
W20260812 03:12:38.943563 17837 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e77b251a801a411aaacc16ee7382bbe7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:12:38.943675 17837 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:12:38.943825 17838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:12:38.944693 17838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:12:38.949443 17815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:38.938662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:56993 (local address 127.0.0.1:38231)
0812 03:12:38.938802 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:38.938807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:38.939250 (+ 443us) server_negotiation.cc:407] Connection header received
0812 03:12:38.939489 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:38.939495 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:38.939561 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:38.939628 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:38.940830 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:38.941523 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:38.942870 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:38.943248 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:38.943383 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:38.945340 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:38.945345 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:38.945348 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:38.945378 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:38.947364 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:38.947944 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:38.947948 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:38.947950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:38.948043 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:38.948427 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:38.948431 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:38.948433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:38.948639 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:38.948823 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:38.949124 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 03:12:38.949258 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:12:38.952219 17838 catalog_manager.cc:1347] Generated new cluster ID: e5ae98cf03b245af82ce32eda2d104c1
I20260812 03:12:38.952301 17838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:12:38.969682 17838 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:12:38.971087 17838 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:12:38.977054 17838 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e77b251a801a411aaacc16ee7382bbe7: Generated new TSK 0
I20260812 03:12:38.977974 17838 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:12:38.982548 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 17741
W20260812 03:12:38.990983 17720 connection.cc:535] client connection to 127.0.0.1:38231 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38231): Connection reset by peer (error 104)
2026-08-12T03:12:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1667 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:12:39 dist-test-slave-zpfg krb5kdc[17846](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:12:39 dist-test-slave-zpfg krb5kdc[17846](info): set up 2 sockets
Aug 12 03:12:39 dist-test-slave-zpfg krb5kdc[17846](info): commencing operation
krb5kdc: starting...
W20260812 03:12:40.066255 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:12:40 dist-test-slave-zpfg krb5kdc[17846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:12:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:12:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:12:40.211517 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41413
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:40.339774 17862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:40.340106 17862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:40.340217 17862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:40.344549 17862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:40.344674 17862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:40.344713 17862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:40.344755 17862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:40.344791 17862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:40.350692 17862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41413
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:40.351944 17862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:40.353047 17862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:40.362332 17870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:40.362370 17869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:40.362394 17862 server_base.cc:1034] running on GCE node
W20260812 03:12:40.362419 17872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:40.363271 17862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:40.364511 17862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:40.365711 17862 hybrid_clock.cc:648] HybridClock initialized: now 1786504360365696 us; error 46 us; skew 500 ppm
Aug 12 03:12:40 dist-test-slave-zpfg krb5kdc[17846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504360, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:40.368860 17862 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:12:40.369874 17862 webserver.cc:460] Webserver started at http://127.0.0.1:46183/ using document root <none> and password file <none>
I20260812 03:12:40.370508 17862 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:40.370580 17862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:40.370774 17862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:40.373080 17862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "2d744349755b405b8f3c54ba9a35b7ab"
format_stamp: "Formatted at 2026-08-12 03:12:40 on dist-test-slave-zpfg"
server_key: "f9720a5783e661bfc594028b47b00c64"
server_key_iv: "14a31d9c34791af1474bcbab00d167e3"
server_key_version: "encryptionkey@0"
I20260812 03:12:40.373631 17862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "2d744349755b405b8f3c54ba9a35b7ab"
format_stamp: "Formatted at 2026-08-12 03:12:40 on dist-test-slave-zpfg"
server_key: "f9720a5783e661bfc594028b47b00c64"
server_key_iv: "14a31d9c34791af1474bcbab00d167e3"
server_key_version: "encryptionkey@0"
I20260812 03:12:40.377714 17862 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260812 03:12:40.380549 17881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:40.382058 17862 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:12:40.382225 17862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "2d744349755b405b8f3c54ba9a35b7ab"
format_stamp: "Formatted at 2026-08-12 03:12:40 on dist-test-slave-zpfg"
server_key: "f9720a5783e661bfc594028b47b00c64"
server_key_iv: "14a31d9c34791af1474bcbab00d167e3"
server_key_version: "encryptionkey@0"
I20260812 03:12:40.382393 17862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:40.395593 17862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:40.399473 17862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:40.399749 17862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:40.408748 17862 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41413
I20260812 03:12:40.408767 17935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41413 every 8 connection(s)
I20260812 03:12:40.409845 17862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:12:40.410686 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 17862
I20260812 03:12:40.410840 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:12:40.411211 27626 external_mini_cluster.cc:1428] Setting key d358207da9cc4b95efbe28a16d9a264e
I20260812 03:12:40.413331 17936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:40.420195 17936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d744349755b405b8f3c54ba9a35b7ab: Bootstrap starting.
I20260812 03:12:40.421234 17939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:40.412837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:46175 (local address 127.0.0.1:41413)
0812 03:12:40.413320 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:40.413343 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:40.413381 (+ 38us) server_negotiation.cc:407] Connection header received
0812 03:12:40.414133 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:40.414151 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:40.414612 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:40.414930 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:40.415290 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:40.416433 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:40.417688 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:40.418214 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:40.418425 (+ 211us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:40.419460 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:40.419465 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:40.419478 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:40.419887 (+ 409us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:40.419943 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:40.419965 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:40.420082 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:40.420340 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 03:12:40.420487 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":160,"threads_started":1}
I20260812 03:12:40.422721 17936 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d744349755b405b8f3c54ba9a35b7ab: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:40.423594 17936 log.cc:824] T 00000000000000000000000000000000 P 2d744349755b405b8f3c54ba9a35b7ab: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:40.424587 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:12:40.425812 17936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d744349755b405b8f3c54ba9a35b7ab: No bootstrap required, opened a new log
I20260812 03:12:40.430403 17936 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d744349755b405b8f3c54ba9a35b7ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d744349755b405b8f3c54ba9a35b7ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41413 } }
I20260812 03:12:40.430728 17936 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d744349755b405b8f3c54ba9a35b7ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:40.430826 17936 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d744349755b405b8f3c54ba9a35b7ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d744349755b405b8f3c54ba9a35b7ab, State: Initialized, Role: FOLLOWER
I20260812 03:12:40.431463 17936 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d744349755b405b8f3c54ba9a35b7ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d744349755b405b8f3c54ba9a35b7ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41413 } }
I20260812 03:12:40.431664 17936 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d744349755b405b8f3c54ba9a35b7ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:12:40.431779 17936 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d744349755b405b8f3c54ba9a35b7ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:12:40.431907 17936 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d744349755b405b8f3c54ba9a35b7ab [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:40.432155 17939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:40.426811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:38743 (local address 127.0.0.1:41413)
0812 03:12:40.426989 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:40.426993 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:40.427227 (+ 234us) server_negotiation.cc:407] Connection header received
0812 03:12:40.427312 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:40.427315 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:40.427364 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:40.427451 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:40.427695 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:40.429162 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:40.429996 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:40.430374 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:40.430497 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:40.431429 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:40.431433 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:40.431435 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:40.431449 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:40.431481 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:40.431496 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:40.431597 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:40.431962 (+ 365us) server_negotiation.cc:299] Negotiation successful
0812 03:12:40.432011 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 03:12:40.433068 17936 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d744349755b405b8f3c54ba9a35b7ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d744349755b405b8f3c54ba9a35b7ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41413 } }
I20260812 03:12:40.433465 17936 leader_election.cc:302] T 00000000000000000000000000000000 P 2d744349755b405b8f3c54ba9a35b7ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d744349755b405b8f3c54ba9a35b7ab; no voters:
I20260812 03:12:40.433841 17936 leader_election.cc:288] T 00000000000000000000000000000000 P 2d744349755b405b8f3c54ba9a35b7ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:12:40.434026 17946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d744349755b405b8f3c54ba9a35b7ab [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:40.434302 17946 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d744349755b405b8f3c54ba9a35b7ab [term 1 LEADER]: Becoming Leader. State: Replica: 2d744349755b405b8f3c54ba9a35b7ab, State: Running, Role: LEADER
I20260812 03:12:40.434767 17946 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d744349755b405b8f3c54ba9a35b7ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d744349755b405b8f3c54ba9a35b7ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41413 } }
I20260812 03:12:40.435312 17936 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d744349755b405b8f3c54ba9a35b7ab [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:40.436558 17948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d744349755b405b8f3c54ba9a35b7ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d744349755b405b8f3c54ba9a35b7ab. Latest consensus state: current_term: 1 leader_uuid: "2d744349755b405b8f3c54ba9a35b7ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d744349755b405b8f3c54ba9a35b7ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41413 } } }
I20260812 03:12:40.436707 17948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d744349755b405b8f3c54ba9a35b7ab [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:40.436867 17947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d744349755b405b8f3c54ba9a35b7ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d744349755b405b8f3c54ba9a35b7ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d744349755b405b8f3c54ba9a35b7ab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41413 } } }
I20260812 03:12:40.437043 17947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d744349755b405b8f3c54ba9a35b7ab [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:40.437413 17957 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:12:40.441061 17957 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:12:40.447458 17957 catalog_manager.cc:1347] Generated new cluster ID: b498ad0bad694bdbb0a39efa72196a49
I20260812 03:12:40.447608 17957 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:12:40.457749 17957 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:12:40.458846 17957 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:12:40.468530 17957 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d744349755b405b8f3c54ba9a35b7ab: Generated new TSK 0
I20260812 03:12:40.469396 17957 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:12:40.474255 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 17862
W20260812 03:12:40.482947 17844 connection.cc:535] client connection to 127.0.0.1:41413 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41413): Connection reset by peer (error 104)
2026-08-12T03:12:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1491 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:12:40 dist-test-slave-zpfg krb5kdc[17968](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:12:40 dist-test-slave-zpfg krb5kdc[17968](info): set up 2 sockets
Aug 12 03:12:40 dist-test-slave-zpfg krb5kdc[17968](info): commencing operation
krb5kdc: starting...
W20260812 03:12:41.611516 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:12:41 dist-test-slave-zpfg krb5kdc[17968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:12:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:12:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 03:12:41.755337 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35945
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:41.879176 17983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:41.879477 17983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:41.879523 17983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:41.883751 17983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:41.883832 17983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:41.883852 17983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:41.883870 17983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:41.883886 17983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:41.889490 17983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35945
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:41.890794 17983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:41.891893 17983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:41.901592 17989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:41.901592 17988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:41.901922 17983 server_base.cc:1034] running on GCE node
W20260812 03:12:41.901681 17991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:41.902798 17983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:41.904258 17983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:41.905488 17983 hybrid_clock.cc:648] HybridClock initialized: now 1786504361905472 us; error 35 us; skew 500 ppm
Aug 12 03:12:41 dist-test-slave-zpfg krb5kdc[17968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504361, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:41.909070 17983 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 03:12:41.910147 17983 webserver.cc:460] Webserver started at http://127.0.0.1:46859/ using document root <none> and password file <none>
I20260812 03:12:41.910907 17983 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:41.911015 17983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:41.911290 17983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:41.913592 17983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "dcf4e2b3bd3d4254b567c27848ad2438"
format_stamp: "Formatted at 2026-08-12 03:12:41 on dist-test-slave-zpfg"
server_key: "c313ff9ef5a7db717247522144d6c299"
server_key_iv: "18b351557d597458a3be5df7052b1946"
server_key_version: "encryptionkey@0"
I20260812 03:12:41.914232 17983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "dcf4e2b3bd3d4254b567c27848ad2438"
format_stamp: "Formatted at 2026-08-12 03:12:41 on dist-test-slave-zpfg"
server_key: "c313ff9ef5a7db717247522144d6c299"
server_key_iv: "18b351557d597458a3be5df7052b1946"
server_key_version: "encryptionkey@0"
I20260812 03:12:41.918824 17983 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 03:12:41.921792 17999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:41.923414 17983 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:12:41.923590 17983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "dcf4e2b3bd3d4254b567c27848ad2438"
format_stamp: "Formatted at 2026-08-12 03:12:41 on dist-test-slave-zpfg"
server_key: "c313ff9ef5a7db717247522144d6c299"
server_key_iv: "18b351557d597458a3be5df7052b1946"
server_key_version: "encryptionkey@0"
I20260812 03:12:41.923738 17983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:41.929929 17983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:41.934698 17983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:41.934930 17983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:41.943269 17983 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35945
I20260812 03:12:41.943478 18056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35945 every 8 connection(s)
I20260812 03:12:41.944432 17983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:12:41.947600 18057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:41.953544 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 17983
I20260812 03:12:41.953666 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:12:41.953658 18057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcf4e2b3bd3d4254b567c27848ad2438: Bootstrap starting.
I20260812 03:12:41.954025 27626 external_mini_cluster.cc:1428] Setting key e939d5b4df8df15b586d780b6efce8b3
I20260812 03:12:41.956564 18057 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcf4e2b3bd3d4254b567c27848ad2438: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:41.957348 18057 log.cc:824] T 00000000000000000000000000000000 P dcf4e2b3bd3d4254b567c27848ad2438: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:41.959831 18057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcf4e2b3bd3d4254b567c27848ad2438: No bootstrap required, opened a new log
I20260812 03:12:41.960129 18060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:41.955682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:60427 (local address 127.0.0.1:35945)
0812 03:12:41.956192 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:41.956207 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:41.956246 (+ 39us) server_negotiation.cc:407] Connection header received
0812 03:12:41.957032 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:41.957052 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:41.957361 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:41.957744 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:41.958613 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:41.958618 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:41.958632 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:41.958642 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:41.958703 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:41.958727 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:41.958862 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:41.959013 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 03:12:41.959174 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":160,"threads_started":1}
I20260812 03:12:41.963092 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:12:41.963968 18057 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcf4e2b3bd3d4254b567c27848ad2438 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf4e2b3bd3d4254b567c27848ad2438" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35945 } }
I20260812 03:12:41.964274 18057 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcf4e2b3bd3d4254b567c27848ad2438 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:41.964344 18057 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcf4e2b3bd3d4254b567c27848ad2438 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcf4e2b3bd3d4254b567c27848ad2438, State: Initialized, Role: FOLLOWER
I20260812 03:12:41.964891 18057 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcf4e2b3bd3d4254b567c27848ad2438 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf4e2b3bd3d4254b567c27848ad2438" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35945 } }
I20260812 03:12:41.965057 18057 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcf4e2b3bd3d4254b567c27848ad2438 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:12:41.965137 18057 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcf4e2b3bd3d4254b567c27848ad2438 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:12:41.965253 18057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcf4e2b3bd3d4254b567c27848ad2438 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:41.966457 18057 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcf4e2b3bd3d4254b567c27848ad2438 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf4e2b3bd3d4254b567c27848ad2438" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35945 } }
I20260812 03:12:41.966809 18057 leader_election.cc:302] T 00000000000000000000000000000000 P dcf4e2b3bd3d4254b567c27848ad2438 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcf4e2b3bd3d4254b567c27848ad2438; no voters:
I20260812 03:12:41.967173 18057 leader_election.cc:288] T 00000000000000000000000000000000 P dcf4e2b3bd3d4254b567c27848ad2438 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:12:41.967237 18067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcf4e2b3bd3d4254b567c27848ad2438 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:41.967437 18060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:41.965491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.67:56383 (local address 127.0.0.1:35945)
0812 03:12:41.965621 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:41.965626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:41.965821 (+ 195us) server_negotiation.cc:407] Connection header received
0812 03:12:41.966013 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:41.966016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:41.966050 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:41.966140 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:41.966952 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:41.966956 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:41.966959 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:41.966970 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:41.966999 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:41.967018 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:41.967112 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:41.967223 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 03:12:41.967267 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 03:12:41.967613 18067 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcf4e2b3bd3d4254b567c27848ad2438 [term 1 LEADER]: Becoming Leader. State: Replica: dcf4e2b3bd3d4254b567c27848ad2438, State: Running, Role: LEADER
I20260812 03:12:41.967929 18067 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcf4e2b3bd3d4254b567c27848ad2438 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf4e2b3bd3d4254b567c27848ad2438" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35945 } }
I20260812 03:12:41.968607 18057 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcf4e2b3bd3d4254b567c27848ad2438 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:41.970041 18069 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcf4e2b3bd3d4254b567c27848ad2438 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcf4e2b3bd3d4254b567c27848ad2438. Latest consensus state: current_term: 1 leader_uuid: "dcf4e2b3bd3d4254b567c27848ad2438" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf4e2b3bd3d4254b567c27848ad2438" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35945 } } }
I20260812 03:12:41.970105 18068 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcf4e2b3bd3d4254b567c27848ad2438 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcf4e2b3bd3d4254b567c27848ad2438" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf4e2b3bd3d4254b567c27848ad2438" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35945 } } }
I20260812 03:12:41.970263 18069 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcf4e2b3bd3d4254b567c27848ad2438 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:41.970293 18068 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcf4e2b3bd3d4254b567c27848ad2438 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:41.971517 18077 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:12:41.974887 18077 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:12:41.981588 18077 catalog_manager.cc:1347] Generated new cluster ID: 1b17aadca51847869d8f705bf83759ad
I20260812 03:12:41.981743 18077 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:12:41.997952 18077 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:12:41.998374 18077 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:12:42.007278 18077 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcf4e2b3bd3d4254b567c27848ad2438: Generated new TSK 0
I20260812 03:12:42.008183 18077 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:12:42.017329 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 17983
2026-08-12T03:12:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1543 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:12:42 dist-test-slave-zpfg krb5kdc[18089](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:12:42 dist-test-slave-zpfg krb5kdc[18089](info): set up 2 sockets
Aug 12 03:12:42 dist-test-slave-zpfg krb5kdc[18089](info): commencing operation
krb5kdc: starting...
W20260812 03:12:43.131861 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:12:43 dist-test-slave-zpfg krb5kdc[18089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:12:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:12:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:12:43.282814 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:40007
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:44807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:40007
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:43.406710 18103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:43.407079 18103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:43.407171 18103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:43.411348 18103 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:43.411465 18103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:43.411505 18103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:43.411543 18103 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:43.411576 18103 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:43.417274 18103 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:44807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:40007
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:40007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:43.418926 18103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:43.420082 18103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:43.430284 18108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:43.430532 18103 server_base.cc:1034] running on GCE node
W20260812 03:12:43.430326 18111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:43.430378 18109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:43.431391 18103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:43.432704 18103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:43.433928 18103 hybrid_clock.cc:648] HybridClock initialized: now 1786504363433907 us; error 34 us; skew 500 ppm
Aug 12 03:12:43 dist-test-slave-zpfg krb5kdc[18089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504363, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:43.437414 18103 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:12:43.438589 18103 webserver.cc:460] Webserver started at http://127.26.250.190:38485/ using document root <none> and password file <none>
I20260812 03:12:43.439208 18103 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:43.439306 18103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:43.439561 18103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:43.441830 18103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "5d4e40d8786745e7a6a36bd8e6c79a1a"
format_stamp: "Formatted at 2026-08-12 03:12:43 on dist-test-slave-zpfg"
server_key: "2a8e0a42236d1136b050acc8143fe8cf"
server_key_iv: "5d668e84e0c5b67b1cf5a044eca82889"
server_key_version: "encryptionkey@0"
I20260812 03:12:43.442551 18103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "5d4e40d8786745e7a6a36bd8e6c79a1a"
format_stamp: "Formatted at 2026-08-12 03:12:43 on dist-test-slave-zpfg"
server_key: "2a8e0a42236d1136b050acc8143fe8cf"
server_key_iv: "5d668e84e0c5b67b1cf5a044eca82889"
server_key_version: "encryptionkey@0"
I20260812 03:12:43.446794 18103 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 03:12:43.449723 18118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:43.451279 18103 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:12:43.451447 18103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "5d4e40d8786745e7a6a36bd8e6c79a1a"
format_stamp: "Formatted at 2026-08-12 03:12:43 on dist-test-slave-zpfg"
server_key: "2a8e0a42236d1136b050acc8143fe8cf"
server_key_iv: "5d668e84e0c5b67b1cf5a044eca82889"
server_key_version: "encryptionkey@0"
I20260812 03:12:43.451588 18103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:43.477496 18103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:43.481195 18103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:43.481441 18103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:43.489892 18103 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:40007
I20260812 03:12:43.489923 18173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:40007 every 8 connection(s)
I20260812 03:12:43.491011 18103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:12:43.491931 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 18103
I20260812 03:12:43.492085 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:12:43.492445 27626 external_mini_cluster.cc:1428] Setting key 00a4206809473b1c9a7a86e23e15c2e5
I20260812 03:12:43.494804 18174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:43.502676 18174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d4e40d8786745e7a6a36bd8e6c79a1a: Bootstrap starting.
Aug 12 03:12:43 dist-test-slave-zpfg krb5kdc[18089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:12:43.505824 18174 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d4e40d8786745e7a6a36bd8e6c79a1a: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:43.506868 18174 log.cc:824] T 00000000000000000000000000000000 P 5d4e40d8786745e7a6a36bd8e6c79a1a: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:43.509392 18174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d4e40d8786745e7a6a36bd8e6c79a1a: No bootstrap required, opened a new log
I20260812 03:12:43.509588 18177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:43.494088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50493 (local address 127.26.250.190:40007)
0812 03:12:43.494625 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:43.494655 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:43.494694 (+ 39us) server_negotiation.cc:407] Connection header received
0812 03:12:43.495416 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:43.495435 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:43.495799 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:43.496103 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:43.497195 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:43.498370 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:43.499642 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:43.500171 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:43.500309 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:43.503596 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:43.503603 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:43.503621 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:43.503661 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:43.506005 (+ 2344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:43.506724 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:43.506734 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:43.506742 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:43.506901 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:43.507321 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:43.507325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:43.507327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:43.507815 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:43.508046 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:43.508326 (+ 280us) server_negotiation.cc:299] Negotiation successful
0812 03:12:43.508575 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":238,"threads_started":1}
I20260812 03:12:43.512960 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:12:43.513929 18174 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d4e40d8786745e7a6a36bd8e6c79a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d4e40d8786745e7a6a36bd8e6c79a1a" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 40007 } }
I20260812 03:12:43.514290 18174 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d4e40d8786745e7a6a36bd8e6c79a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:43.514408 18174 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d4e40d8786745e7a6a36bd8e6c79a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4e40d8786745e7a6a36bd8e6c79a1a, State: Initialized, Role: FOLLOWER
I20260812 03:12:43.514915 18174 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d4e40d8786745e7a6a36bd8e6c79a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d4e40d8786745e7a6a36bd8e6c79a1a" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 40007 } }
I20260812 03:12:43.515092 18174 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d4e40d8786745e7a6a36bd8e6c79a1a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:12:43.515211 18174 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d4e40d8786745e7a6a36bd8e6c79a1a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:12:43.515363 18174 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d4e40d8786745e7a6a36bd8e6c79a1a [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:43.516886 18174 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d4e40d8786745e7a6a36bd8e6c79a1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d4e40d8786745e7a6a36bd8e6c79a1a" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 40007 } }
I20260812 03:12:43.517280 18174 leader_election.cc:302] T 00000000000000000000000000000000 P 5d4e40d8786745e7a6a36bd8e6c79a1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d4e40d8786745e7a6a36bd8e6c79a1a; no voters:
I20260812 03:12:43.517639 18174 leader_election.cc:288] T 00000000000000000000000000000000 P 5d4e40d8786745e7a6a36bd8e6c79a1a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:12:43.517732 18185 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d4e40d8786745e7a6a36bd8e6c79a1a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:43.517930 18185 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d4e40d8786745e7a6a36bd8e6c79a1a [term 1 LEADER]: Becoming Leader. State: Replica: 5d4e40d8786745e7a6a36bd8e6c79a1a, State: Running, Role: LEADER
I20260812 03:12:43.518323 18185 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d4e40d8786745e7a6a36bd8e6c79a1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d4e40d8786745e7a6a36bd8e6c79a1a" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 40007 } }
I20260812 03:12:43.519135 18174 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d4e40d8786745e7a6a36bd8e6c79a1a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:43.520406 18187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d4e40d8786745e7a6a36bd8e6c79a1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d4e40d8786745e7a6a36bd8e6c79a1a. Latest consensus state: current_term: 1 leader_uuid: "5d4e40d8786745e7a6a36bd8e6c79a1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d4e40d8786745e7a6a36bd8e6c79a1a" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 40007 } } }
I20260812 03:12:43.520555 18187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d4e40d8786745e7a6a36bd8e6c79a1a [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:43.520885 18186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d4e40d8786745e7a6a36bd8e6c79a1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d4e40d8786745e7a6a36bd8e6c79a1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d4e40d8786745e7a6a36bd8e6c79a1a" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 40007 } } }
I20260812 03:12:43.521000 18186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d4e40d8786745e7a6a36bd8e6c79a1a [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:43.521798 18195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:12:43.525285 18195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:12:43.525315 18177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:43.515280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35125 (local address 127.26.250.190:40007)
0812 03:12:43.515430 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:43.515434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:43.515595 (+ 161us) server_negotiation.cc:407] Connection header received
0812 03:12:43.515736 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:43.515738 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:43.515781 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:43.515853 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:43.516723 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:43.517345 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:43.518534 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:43.518779 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:43.518901 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:43.520873 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:43.520878 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:43.520883 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:43.520912 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:43.522671 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:43.523300 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:43.523305 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:43.523307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:43.523400 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:43.523782 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:43.523785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:43.523787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:43.524028 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:43.524181 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:43.525049 (+ 868us) server_negotiation.cc:299] Negotiation successful
0812 03:12:43.525156 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 03:12:43.533306 18195 catalog_manager.cc:1347] Generated new cluster ID: fe13466b938442b697d59480739ca8b7
I20260812 03:12:43.533414 18195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:12:43.558261 18195 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:12:43.559612 18195 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:12:43.581575 18195 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d4e40d8786745e7a6a36bd8e6c79a1a: Generated new TSK 0
I20260812 03:12:43.582615 18195 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:12:43.594300 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 18103
W20260812 03:12:43.602880 18087 connection.cc:535] client connection to 127.26.250.190:40007 recv error: Network error: failed to read from TLS socket (remote: 127.26.250.190:40007): Connection reset by peer (error 104)
2026-08-12T03:12:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1576 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:12:43 dist-test-slave-zpfg krb5kdc[18207](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:12:43 dist-test-slave-zpfg krb5kdc[18207](info): set up 2 sockets
Aug 12 03:12:43 dist-test-slave-zpfg krb5kdc[18207](info): commencing operation
krb5kdc: starting...
W20260812 03:12:44.711336 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:12:44 dist-test-slave-zpfg krb5kdc[18207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:12:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:12:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:12:44.858737 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:42167
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:42979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:42167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:44.978538 18221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:44.978899 18221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:44.978978 18221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:44.983289 18221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:44.983412 18221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:44.983446 18221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:44.983486 18221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:44.983520 18221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:44.989233 18221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:42979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:42167
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:42167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:44.990487 18221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:44.991601 18221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:45.001008 18228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:45.001031 18230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:45.001032 18227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:45.001435 18221 server_base.cc:1034] running on GCE node
I20260812 03:12:45.002197 18221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:45.003575 18221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:45.004841 18221 hybrid_clock.cc:648] HybridClock initialized: now 1786504365004819 us; error 36 us; skew 500 ppm
Aug 12 03:12:45 dist-test-slave-zpfg krb5kdc[18207](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504365, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:45.014204 18221 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:12:45.015713 18221 webserver.cc:460] Webserver started at http://127.26.250.190:45883/ using document root <none> and password file <none>
I20260812 03:12:45.016564 18221 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:45.016644 18221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:45.016907 18221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:45.019348 18221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "349f8c4828334a98b4a53baa4b42b871"
format_stamp: "Formatted at 2026-08-12 03:12:45 on dist-test-slave-zpfg"
server_key: "df8a237248ad92b51f8be1e39ac33fd7"
server_key_iv: "3272d23465e50909abf117d84cb7dc86"
server_key_version: "encryptionkey@0"
I20260812 03:12:45.020026 18221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "349f8c4828334a98b4a53baa4b42b871"
format_stamp: "Formatted at 2026-08-12 03:12:45 on dist-test-slave-zpfg"
server_key: "df8a237248ad92b51f8be1e39ac33fd7"
server_key_iv: "3272d23465e50909abf117d84cb7dc86"
server_key_version: "encryptionkey@0"
I20260812 03:12:45.024466 18221 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:12:45.027354 18237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:45.028875 18221 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:12:45.029037 18221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "349f8c4828334a98b4a53baa4b42b871"
format_stamp: "Formatted at 2026-08-12 03:12:45 on dist-test-slave-zpfg"
server_key: "df8a237248ad92b51f8be1e39ac33fd7"
server_key_iv: "3272d23465e50909abf117d84cb7dc86"
server_key_version: "encryptionkey@0"
I20260812 03:12:45.029175 18221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:45.056542 18221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:45.059864 18221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:45.060091 18221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:45.068207 18221 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:42167
I20260812 03:12:45.068238 18291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:42167 every 8 connection(s)
I20260812 03:12:45.069319 18221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:12:45.072363 18292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:45.077658 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 18221
I20260812 03:12:45.077853 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:12:45.078168 27626 external_mini_cluster.cc:1428] Setting key f5a009586287b89f35a1cbc9b0e915fd
I20260812 03:12:45.079006 18292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 349f8c4828334a98b4a53baa4b42b871: Bootstrap starting.
I20260812 03:12:45.081537 18292 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 349f8c4828334a98b4a53baa4b42b871: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:45.082324 18292 log.cc:824] T 00000000000000000000000000000000 P 349f8c4828334a98b4a53baa4b42b871: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:45.084388 18292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 349f8c4828334a98b4a53baa4b42b871: No bootstrap required, opened a new log
I20260812 03:12:45.088413 18295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:45.079799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42133 (local address 127.26.250.190:42167)
0812 03:12:45.080254 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:45.080271 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:45.080309 (+ 38us) server_negotiation.cc:407] Connection header received
0812 03:12:45.080961 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:45.080978 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:45.081409 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:45.081900 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:45.082461 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:45.083626 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:45.084711 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:45.085247 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:45.085446 (+ 199us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:45.086548 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:45.086554 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:45.086568 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:45.087018 (+ 450us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:45.087077 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:45.087101 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:45.087263 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:45.087428 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 03:12:45.087590 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":154,"threads_started":1}
I20260812 03:12:45.088315 18292 raft_consensus.cc:348] T 00000000000000000000000000000000 P 349f8c4828334a98b4a53baa4b42b871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349f8c4828334a98b4a53baa4b42b871" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42167 } }
I20260812 03:12:45.088580 18292 raft_consensus.cc:374] T 00000000000000000000000000000000 P 349f8c4828334a98b4a53baa4b42b871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:45.088677 18292 raft_consensus.cc:729] T 00000000000000000000000000000000 P 349f8c4828334a98b4a53baa4b42b871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349f8c4828334a98b4a53baa4b42b871, State: Initialized, Role: FOLLOWER
I20260812 03:12:45.089175 18292 consensus_queue.cc:260] T 00000000000000000000000000000000 P 349f8c4828334a98b4a53baa4b42b871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349f8c4828334a98b4a53baa4b42b871" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42167 } }
I20260812 03:12:45.089346 18292 raft_consensus.cc:388] T 00000000000000000000000000000000 P 349f8c4828334a98b4a53baa4b42b871 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:12:45.089422 18292 raft_consensus.cc:482] T 00000000000000000000000000000000 P 349f8c4828334a98b4a53baa4b42b871 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:12:45.090180 18292 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 349f8c4828334a98b4a53baa4b42b871 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:45.091355 18292 raft_consensus.cc:504] T 00000000000000000000000000000000 P 349f8c4828334a98b4a53baa4b42b871 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349f8c4828334a98b4a53baa4b42b871" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42167 } }
I20260812 03:12:45.091763 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:12:45.091771 18292 leader_election.cc:302] T 00000000000000000000000000000000 P 349f8c4828334a98b4a53baa4b42b871 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 349f8c4828334a98b4a53baa4b42b871; no voters:
I20260812 03:12:45.092171 18292 leader_election.cc:288] T 00000000000000000000000000000000 P 349f8c4828334a98b4a53baa4b42b871 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:12:45.092243 18297 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 349f8c4828334a98b4a53baa4b42b871 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:45.092492 18297 raft_consensus.cc:686] T 00000000000000000000000000000000 P 349f8c4828334a98b4a53baa4b42b871 [term 1 LEADER]: Becoming Leader. State: Replica: 349f8c4828334a98b4a53baa4b42b871, State: Running, Role: LEADER
I20260812 03:12:45.092828 18297 consensus_queue.cc:237] T 00000000000000000000000000000000 P 349f8c4828334a98b4a53baa4b42b871 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349f8c4828334a98b4a53baa4b42b871" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42167 } }
I20260812 03:12:45.093591 18292 sys_catalog.cc:564] T 00000000000000000000000000000000 P 349f8c4828334a98b4a53baa4b42b871 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:45.094750 18299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 349f8c4828334a98b4a53baa4b42b871 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "349f8c4828334a98b4a53baa4b42b871" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349f8c4828334a98b4a53baa4b42b871" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42167 } } }
I20260812 03:12:45.094786 18298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 349f8c4828334a98b4a53baa4b42b871 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 349f8c4828334a98b4a53baa4b42b871. Latest consensus state: current_term: 1 leader_uuid: "349f8c4828334a98b4a53baa4b42b871" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349f8c4828334a98b4a53baa4b42b871" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42167 } } }
I20260812 03:12:45.094905 18298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 349f8c4828334a98b4a53baa4b42b871 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:45.094907 18299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 349f8c4828334a98b4a53baa4b42b871 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:45.095320 18310 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:12:45.098703 18310 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:12:45.103173 18295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:45.094633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35173 (local address 127.26.250.190:42167)
0812 03:12:45.094774 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:45.094780 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:45.098682 (+ 3902us) server_negotiation.cc:407] Connection header received
0812 03:12:45.098880 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:45.098887 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:45.098946 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:45.099005 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:45.099530 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:45.100076 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:45.101026 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:45.101398 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:45.101518 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:45.102585 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:45.102590 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:45.102595 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:45.102612 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:45.102647 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:45.102662 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:45.102766 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:45.102976 (+ 210us) server_negotiation.cc:299] Negotiation successful
0812 03:12:45.103019 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:12:45.105372 18310 catalog_manager.cc:1347] Generated new cluster ID: 69d3b8372da84d66861f38ab1e8988ae
I20260812 03:12:45.105466 18310 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:12:45.113090 18310 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:12:45.114434 18310 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:12:45.126526 18310 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 349f8c4828334a98b4a53baa4b42b871: Generated new TSK 0
I20260812 03:12:45.127417 18310 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:12:45.133229 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 18221
W20260812 03:12:45.141043 18205 connection.cc:535] client connection to 127.26.250.190:42167 recv error: Network error: failed to read from TLS socket (remote: 127.26.250.190:42167): Connection reset by peer (error 104)
2026-08-12T03:12:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1538 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:12:45 dist-test-slave-zpfg krb5kdc[18325](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:12:45 dist-test-slave-zpfg krb5kdc[18325](info): set up 2 sockets
Aug 12 03:12:45 dist-test-slave-zpfg krb5kdc[18325](info): commencing operation
krb5kdc: starting...
W20260812 03:12:46.236009 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:12:46 dist-test-slave-zpfg krb5kdc[18325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:12:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:12:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:12:46.381850 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:42025
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:33567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:42025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:46.500734 18339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:46.501078 18339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:46.501170 18339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:46.505366 18339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:46.505475 18339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:46.505517 18339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:46.505553 18339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:46.505584 18339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:46.511051 18339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:33567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:42025
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:42025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:46.512295 18339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:46.513433 18339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:46.522730 18346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:46.522775 18349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:46.522775 18345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:46.522980 18339 server_base.cc:1034] running on GCE node
I20260812 03:12:46.523706 18339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:46.524968 18339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:46.526139 18339 hybrid_clock.cc:648] HybridClock initialized: now 1786504366526118 us; error 35 us; skew 500 ppm
Aug 12 03:12:46 dist-test-slave-zpfg krb5kdc[18325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504366, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:46.529161 18339 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:12:46.530161 18339 webserver.cc:460] Webserver started at http://127.26.250.190:46431/ using document root <none> and password file <none>
I20260812 03:12:46.530838 18339 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:46.530910 18339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:46.531155 18339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:46.533337 18339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "fb14268194ee4069be8aee01b15a27ab"
format_stamp: "Formatted at 2026-08-12 03:12:46 on dist-test-slave-zpfg"
server_key: "2ca84a5afbc2a2eaa33b3941d1dbbc45"
server_key_iv: "c346aa8a6c7a86b5c44e517afa809a74"
server_key_version: "encryptionkey@0"
I20260812 03:12:46.533939 18339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "fb14268194ee4069be8aee01b15a27ab"
format_stamp: "Formatted at 2026-08-12 03:12:46 on dist-test-slave-zpfg"
server_key: "2ca84a5afbc2a2eaa33b3941d1dbbc45"
server_key_iv: "c346aa8a6c7a86b5c44e517afa809a74"
server_key_version: "encryptionkey@0"
I20260812 03:12:46.538108 18339 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 03:12:46.540699 18357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:46.542083 18339 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 03:12:46.542251 18339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "fb14268194ee4069be8aee01b15a27ab"
format_stamp: "Formatted at 2026-08-12 03:12:46 on dist-test-slave-zpfg"
server_key: "2ca84a5afbc2a2eaa33b3941d1dbbc45"
server_key_iv: "c346aa8a6c7a86b5c44e517afa809a74"
server_key_version: "encryptionkey@0"
I20260812 03:12:46.542402 18339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:46.547201 18339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:46.550556 18339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:46.550760 18339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:46.558676 18339 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:42025
I20260812 03:12:46.558706 18413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:42025 every 8 connection(s)
I20260812 03:12:46.559651 18339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:12:46.562618 18414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:46.568616 18414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb14268194ee4069be8aee01b15a27ab: Bootstrap starting.
I20260812 03:12:46.569897 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 18339
I20260812 03:12:46.570052 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:12:46.570477 27626 external_mini_cluster.cc:1428] Setting key 06826070d1e888c08911136bfbf1966f
I20260812 03:12:46.571537 18414 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb14268194ee4069be8aee01b15a27ab: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:46.572395 18414 log.cc:824] T 00000000000000000000000000000000 P fb14268194ee4069be8aee01b15a27ab: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:46.574613 18414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb14268194ee4069be8aee01b15a27ab: No bootstrap required, opened a new log
I20260812 03:12:46.575906 18417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:46.571673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40601 (local address 127.26.250.190:42025)
0812 03:12:46.572171 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:46.572187 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:46.572223 (+ 36us) server_negotiation.cc:407] Connection header received
0812 03:12:46.573103 (+ 880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:46.573133 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:46.573454 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:46.573806 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:46.574670 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:46.574678 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:46.574693 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:46.574708 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:46.574780 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:46.574802 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:46.574924 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:46.575026 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 03:12:46.575187 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":153,"threads_started":1}
I20260812 03:12:46.578503 18414 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb14268194ee4069be8aee01b15a27ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb14268194ee4069be8aee01b15a27ab" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42025 } }
I20260812 03:12:46.578728 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:12:46.578784 18414 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb14268194ee4069be8aee01b15a27ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:46.578881 18414 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb14268194ee4069be8aee01b15a27ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb14268194ee4069be8aee01b15a27ab, State: Initialized, Role: FOLLOWER
I20260812 03:12:46.579380 18414 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb14268194ee4069be8aee01b15a27ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb14268194ee4069be8aee01b15a27ab" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42025 } }
I20260812 03:12:46.579510 18414 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb14268194ee4069be8aee01b15a27ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:12:46.579597 18414 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb14268194ee4069be8aee01b15a27ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:12:46.579701 18414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb14268194ee4069be8aee01b15a27ab [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:46.580888 18414 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb14268194ee4069be8aee01b15a27ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb14268194ee4069be8aee01b15a27ab" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42025 } }
I20260812 03:12:46.581281 18414 leader_election.cc:302] T 00000000000000000000000000000000 P fb14268194ee4069be8aee01b15a27ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb14268194ee4069be8aee01b15a27ab; no voters:
I20260812 03:12:46.581678 18414 leader_election.cc:288] T 00000000000000000000000000000000 P fb14268194ee4069be8aee01b15a27ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:12:46.581723 18425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb14268194ee4069be8aee01b15a27ab [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:46.582250 18425 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb14268194ee4069be8aee01b15a27ab [term 1 LEADER]: Becoming Leader. State: Replica: fb14268194ee4069be8aee01b15a27ab, State: Running, Role: LEADER
I20260812 03:12:46.582641 18425 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb14268194ee4069be8aee01b15a27ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb14268194ee4069be8aee01b15a27ab" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42025 } }
I20260812 03:12:46.582890 18417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:46.580995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36921 (local address 127.26.250.190:42025)
0812 03:12:46.581111 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:46.581115 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:46.581312 (+ 197us) server_negotiation.cc:407] Connection header received
0812 03:12:46.581464 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:46.581467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:46.581501 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:46.581590 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:46.582448 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:46.582452 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:46.582455 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:46.582467 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:46.582494 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:46.582514 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:46.582570 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:46.582722 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 03:12:46.582762 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:12:46.583097 18414 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb14268194ee4069be8aee01b15a27ab [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:46.584338 18426 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb14268194ee4069be8aee01b15a27ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb14268194ee4069be8aee01b15a27ab. Latest consensus state: current_term: 1 leader_uuid: "fb14268194ee4069be8aee01b15a27ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb14268194ee4069be8aee01b15a27ab" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42025 } } }
I20260812 03:12:46.584470 18426 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb14268194ee4069be8aee01b15a27ab [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:46.584831 18427 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb14268194ee4069be8aee01b15a27ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb14268194ee4069be8aee01b15a27ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb14268194ee4069be8aee01b15a27ab" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 42025 } } }
I20260812 03:12:46.584937 18427 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb14268194ee4069be8aee01b15a27ab [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:46.584889 18436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:12:46.588068 18436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:12:46.594413 18436 catalog_manager.cc:1347] Generated new cluster ID: 8e2a600c55ef44a7a6bcc757da494d5a
I20260812 03:12:46.594515 18436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:12:46.619503 18436 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:12:46.619916 18436 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:12:46.629923 18436 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb14268194ee4069be8aee01b15a27ab: Generated new TSK 0
I20260812 03:12:46.630849 18436 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:12:46.635385 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 18339
2026-08-12T03:12:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1502 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:12:46 dist-test-slave-zpfg krb5kdc[18448](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:12:46 dist-test-slave-zpfg krb5kdc[18448](info): set up 2 sockets
Aug 12 03:12:46 dist-test-slave-zpfg krb5kdc[18448](info): commencing operation
krb5kdc: starting...
W20260812 03:12:47.739214 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:12:47 dist-test-slave-zpfg krb5kdc[18448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:12:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:12:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:12:47.882612 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:43801
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:35413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:43801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:48.004278 18462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:48.004663 18462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:48.004760 18462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:48.009255 18462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:48.009389 18462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:48.009428 18462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:48.009469 18462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:48.009503 18462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:48.015319 18462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:35413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:43801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:43801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:48.016690 18462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:48.017841 18462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:48.027381 18472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:48.027315 18469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:48.027272 18470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:48.027582 18462 server_base.cc:1034] running on GCE node
I20260812 03:12:48.028277 18462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:48.029577 18462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:48.030773 18462 hybrid_clock.cc:648] HybridClock initialized: now 1786504368030739 us; error 57 us; skew 500 ppm
Aug 12 03:12:48 dist-test-slave-zpfg krb5kdc[18448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504368, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:48.034009 18462 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:12:48.035126 18462 webserver.cc:460] Webserver started at http://127.26.250.190:39033/ using document root <none> and password file <none>
I20260812 03:12:48.035811 18462 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:48.035915 18462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:48.036182 18462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:48.038488 18462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "9b3989a85a22402d95dc05f60f7d383c"
format_stamp: "Formatted at 2026-08-12 03:12:48 on dist-test-slave-zpfg"
server_key: "2e09fff4763ba2ba9baa8357d9222b09"
server_key_iv: "1dd62aac1d2c6842b73215c435e36329"
server_key_version: "encryptionkey@0"
I20260812 03:12:48.039105 18462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "9b3989a85a22402d95dc05f60f7d383c"
format_stamp: "Formatted at 2026-08-12 03:12:48 on dist-test-slave-zpfg"
server_key: "2e09fff4763ba2ba9baa8357d9222b09"
server_key_iv: "1dd62aac1d2c6842b73215c435e36329"
server_key_version: "encryptionkey@0"
I20260812 03:12:48.043195 18462 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 03:12:48.045943 18479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:48.047444 18462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:12:48.047597 18462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "9b3989a85a22402d95dc05f60f7d383c"
format_stamp: "Formatted at 2026-08-12 03:12:48 on dist-test-slave-zpfg"
server_key: "2e09fff4763ba2ba9baa8357d9222b09"
server_key_iv: "1dd62aac1d2c6842b73215c435e36329"
server_key_version: "encryptionkey@0"
I20260812 03:12:48.047724 18462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:48.081444 18462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:48.085175 18462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:48.085423 18462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:48.093467 18462 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:43801
I20260812 03:12:48.093492 18536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:43801 every 8 connection(s)
I20260812 03:12:48.094660 18462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:12:48.097661 18537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:48.101684 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 18462
I20260812 03:12:48.101845 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:12:48.102146 27626 external_mini_cluster.cc:1428] Setting key 0423d5de5c118890b180a97df3080123
I20260812 03:12:48.103998 18537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b3989a85a22402d95dc05f60f7d383c: Bootstrap starting.
I20260812 03:12:48.107007 18537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b3989a85a22402d95dc05f60f7d383c: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:48.108028 18537 log.cc:824] T 00000000000000000000000000000000 P 9b3989a85a22402d95dc05f60f7d383c: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:48.110280 18537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b3989a85a22402d95dc05f60f7d383c: No bootstrap required, opened a new log
Aug 12 03:12:48 dist-test-slave-zpfg krb5kdc[18448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:12:48.115144 18537 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b3989a85a22402d95dc05f60f7d383c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b3989a85a22402d95dc05f60f7d383c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43801 } }
I20260812 03:12:48.115454 18537 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b3989a85a22402d95dc05f60f7d383c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:48.115543 18537 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b3989a85a22402d95dc05f60f7d383c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b3989a85a22402d95dc05f60f7d383c, State: Initialized, Role: FOLLOWER
I20260812 03:12:48.116194 18537 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b3989a85a22402d95dc05f60f7d383c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b3989a85a22402d95dc05f60f7d383c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43801 } }
I20260812 03:12:48.116398 18537 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b3989a85a22402d95dc05f60f7d383c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:12:48.116470 18537 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b3989a85a22402d95dc05f60f7d383c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:12:48.116590 18537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b3989a85a22402d95dc05f60f7d383c [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:48.118008 18537 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b3989a85a22402d95dc05f60f7d383c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b3989a85a22402d95dc05f60f7d383c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43801 } }
I20260812 03:12:48.118425 18537 leader_election.cc:302] T 00000000000000000000000000000000 P 9b3989a85a22402d95dc05f60f7d383c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b3989a85a22402d95dc05f60f7d383c; no voters:
I20260812 03:12:48.118840 18537 leader_election.cc:288] T 00000000000000000000000000000000 P 9b3989a85a22402d95dc05f60f7d383c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:12:48.118995 18542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b3989a85a22402d95dc05f60f7d383c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:48.119268 18542 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b3989a85a22402d95dc05f60f7d383c [term 1 LEADER]: Becoming Leader. State: Replica: 9b3989a85a22402d95dc05f60f7d383c, State: Running, Role: LEADER
I20260812 03:12:48.119431 18540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:48.103893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44089 (local address 127.26.250.190:43801)
0812 03:12:48.104382 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:48.104402 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:48.104446 (+ 44us) server_negotiation.cc:407] Connection header received
0812 03:12:48.105318 (+ 872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:48.105347 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:48.105834 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:48.106213 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:48.107136 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:48.108632 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:48.109610 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:48.110122 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:48.110310 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:48.113467 (+ 3157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:48.113473 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:48.113489 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:48.113519 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:48.115567 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:48.116219 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:48.116226 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:48.116233 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:48.116421 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:48.116781 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:48.116785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:48.116787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:48.117356 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:48.117589 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:48.117927 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 03:12:48.118272 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":167,"threads_started":1}
I20260812 03:12:48.119669 18542 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b3989a85a22402d95dc05f60f7d383c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b3989a85a22402d95dc05f60f7d383c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43801 } }
I20260812 03:12:48.120230 18537 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b3989a85a22402d95dc05f60f7d383c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:48.122406 18544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b3989a85a22402d95dc05f60f7d383c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b3989a85a22402d95dc05f60f7d383c. Latest consensus state: current_term: 1 leader_uuid: "9b3989a85a22402d95dc05f60f7d383c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b3989a85a22402d95dc05f60f7d383c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43801 } } }
I20260812 03:12:48.122571 18544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b3989a85a22402d95dc05f60f7d383c [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:48.122412 18543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b3989a85a22402d95dc05f60f7d383c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b3989a85a22402d95dc05f60f7d383c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b3989a85a22402d95dc05f60f7d383c" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43801 } } }
I20260812 03:12:48.123064 18543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b3989a85a22402d95dc05f60f7d383c [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:48.123140 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:12:48.123309 18557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:12:48.126624 18557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:12:48.133627 18557 catalog_manager.cc:1347] Generated new cluster ID: 44477710b3bc4d2c83f5d72af105e88e
I20260812 03:12:48.133710 18557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:12:48.137987 18540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:48.127475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37483 (local address 127.26.250.190:43801)
0812 03:12:48.127626 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:48.127632 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:48.128296 (+ 664us) server_negotiation.cc:407] Connection header received
0812 03:12:48.128489 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:48.128492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:48.128546 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:48.128654 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:48.129596 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:48.130517 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:48.131748 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:48.132024 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:48.132159 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:48.134021 (+ 1862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:48.134025 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:48.134028 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:48.134059 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:48.135985 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:48.136528 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:48.136532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:48.136534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:48.136643 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:48.136959 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:48.136969 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:48.136971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:48.137215 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:48.137355 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:48.137662 (+ 307us) server_negotiation.cc:299] Negotiation successful
0812 03:12:48.137800 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 03:12:48.140463 18557 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:12:48.141464 18557 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:12:48.146552 18557 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b3989a85a22402d95dc05f60f7d383c: Generated new TSK 0
I20260812 03:12:48.147387 18557 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:12:48.157296 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 18462
2026-08-12T03:12:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1523 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:12:48 dist-test-slave-zpfg krb5kdc[18569](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:12:48 dist-test-slave-zpfg krb5kdc[18569](info): set up 2 sockets
Aug 12 03:12:48 dist-test-slave-zpfg krb5kdc[18569](info): commencing operation
krb5kdc: starting...
W20260812 03:12:49.244352 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:12:49 dist-test-slave-zpfg krb5kdc[18569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:12:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:12:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:12:49.385900 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:38267
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:46761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:38267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:49.503742 18582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:49.504071 18582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:49.504145 18582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:49.508276 18582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:49.508380 18582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:49.508419 18582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:49.508476 18582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:49.508513 18582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:49.513983 18582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:46761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:38267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:38267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:49.515210 18582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:49.516348 18582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:49.525540 18588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:49.525738 18589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:49.525571 18591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:49.526052 18582 server_base.cc:1034] running on GCE node
I20260812 03:12:49.526734 18582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:49.528028 18582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:49.529232 18582 hybrid_clock.cc:648] HybridClock initialized: now 1786504369529217 us; error 30 us; skew 500 ppm
Aug 12 03:12:49 dist-test-slave-zpfg krb5kdc[18569](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504369, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:49.532311 18582 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:12:49.533282 18582 webserver.cc:460] Webserver started at http://127.26.250.190:41271/ using document root <none> and password file <none>
I20260812 03:12:49.533811 18582 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:49.533879 18582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:49.534055 18582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:49.536254 18582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "71ef0c4e88d543d4bf8527c105a69114"
format_stamp: "Formatted at 2026-08-12 03:12:49 on dist-test-slave-zpfg"
server_key: "1b8163ef2bdc3f1184ccfad9134331a9"
server_key_iv: "1ed7d26aa1462289a1b8b9a16ebd1b77"
server_key_version: "encryptionkey@0"
I20260812 03:12:49.536814 18582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "71ef0c4e88d543d4bf8527c105a69114"
format_stamp: "Formatted at 2026-08-12 03:12:49 on dist-test-slave-zpfg"
server_key: "1b8163ef2bdc3f1184ccfad9134331a9"
server_key_iv: "1ed7d26aa1462289a1b8b9a16ebd1b77"
server_key_version: "encryptionkey@0"
I20260812 03:12:49.540978 18582 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:12:49.543720 18598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:49.545138 18582 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 03:12:49.545295 18582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "71ef0c4e88d543d4bf8527c105a69114"
format_stamp: "Formatted at 2026-08-12 03:12:49 on dist-test-slave-zpfg"
server_key: "1b8163ef2bdc3f1184ccfad9134331a9"
server_key_iv: "1ed7d26aa1462289a1b8b9a16ebd1b77"
server_key_version: "encryptionkey@0"
I20260812 03:12:49.545430 18582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:49.572566 18582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:49.576320 18582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:49.576551 18582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:49.584198 18582 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:38267
I20260812 03:12:49.584244 18652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:38267 every 8 connection(s)
I20260812 03:12:49.585330 18582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:12:49.588534 18654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:49.594533 18654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71ef0c4e88d543d4bf8527c105a69114: Bootstrap starting.
I20260812 03:12:49.594563 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 18582
I20260812 03:12:49.594717 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:12:49.595031 27626 external_mini_cluster.cc:1428] Setting key 31ab49c501f6153baee6d0f339691b83
I20260812 03:12:49.597301 18654 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71ef0c4e88d543d4bf8527c105a69114: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:49.598099 18654 log.cc:824] T 00000000000000000000000000000000 P 71ef0c4e88d543d4bf8527c105a69114: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:49.600106 18654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71ef0c4e88d543d4bf8527c105a69114: No bootstrap required, opened a new log
I20260812 03:12:49.603781 18654 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71ef0c4e88d543d4bf8527c105a69114 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ef0c4e88d543d4bf8527c105a69114" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 38267 } }
I20260812 03:12:49.604033 18654 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71ef0c4e88d543d4bf8527c105a69114 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:49.604112 18654 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71ef0c4e88d543d4bf8527c105a69114 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71ef0c4e88d543d4bf8527c105a69114, State: Initialized, Role: FOLLOWER
I20260812 03:12:49.604331 18658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:49.596493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54857 (local address 127.26.250.190:38267)
0812 03:12:49.596978 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:49.596995 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:49.597032 (+ 37us) server_negotiation.cc:407] Connection header received
0812 03:12:49.597752 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:49.597772 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:49.598138 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:49.598563 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:49.598994 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:49.599985 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:49.600943 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:49.601423 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:49.601595 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:49.602522 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:49.602527 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:49.602539 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:49.602935 (+ 396us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:49.602990 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:49.603012 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:49.603125 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:49.603243 (+ 118us) server_negotiation.cc:299] Negotiation successful
0812 03:12:49.603464 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":156,"threads_started":1}
I20260812 03:12:49.604632 18654 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71ef0c4e88d543d4bf8527c105a69114 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ef0c4e88d543d4bf8527c105a69114" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 38267 } }
I20260812 03:12:49.604796 18654 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71ef0c4e88d543d4bf8527c105a69114 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:12:49.604864 18654 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71ef0c4e88d543d4bf8527c105a69114 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:12:49.604959 18654 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71ef0c4e88d543d4bf8527c105a69114 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:49.605988 18654 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71ef0c4e88d543d4bf8527c105a69114 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ef0c4e88d543d4bf8527c105a69114" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 38267 } }
I20260812 03:12:49.606397 18654 leader_election.cc:302] T 00000000000000000000000000000000 P 71ef0c4e88d543d4bf8527c105a69114 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71ef0c4e88d543d4bf8527c105a69114; no voters:
I20260812 03:12:49.606796 18654 leader_election.cc:288] T 00000000000000000000000000000000 P 71ef0c4e88d543d4bf8527c105a69114 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:12:49.606900 18660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71ef0c4e88d543d4bf8527c105a69114 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:49.607172 18660 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71ef0c4e88d543d4bf8527c105a69114 [term 1 LEADER]: Becoming Leader. State: Replica: 71ef0c4e88d543d4bf8527c105a69114, State: Running, Role: LEADER
I20260812 03:12:49.607527 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:12:49.608110 18654 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71ef0c4e88d543d4bf8527c105a69114 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:49.609493 18660 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71ef0c4e88d543d4bf8527c105a69114 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ef0c4e88d543d4bf8527c105a69114" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 38267 } }
I20260812 03:12:49.611523 18662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71ef0c4e88d543d4bf8527c105a69114 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71ef0c4e88d543d4bf8527c105a69114. Latest consensus state: current_term: 1 leader_uuid: "71ef0c4e88d543d4bf8527c105a69114" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ef0c4e88d543d4bf8527c105a69114" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 38267 } } }
I20260812 03:12:49.611518 18661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71ef0c4e88d543d4bf8527c105a69114 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71ef0c4e88d543d4bf8527c105a69114" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71ef0c4e88d543d4bf8527c105a69114" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 38267 } } }
I20260812 03:12:49.611874 18662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71ef0c4e88d543d4bf8527c105a69114 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:49.611894 18661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71ef0c4e88d543d4bf8527c105a69114 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:49.615391 18658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:49.609831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55999 (local address 127.26.250.190:38267)
0812 03:12:49.609941 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:49.609945 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:49.610662 (+ 717us) server_negotiation.cc:407] Connection header received
0812 03:12:49.611477 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:49.611483 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:49.611540 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:49.611623 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:49.612069 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:49.612818 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:49.613664 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:49.613917 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:49.614030 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:49.614935 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:49.614939 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:49.614942 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:49.614955 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:49.614990 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:49.615002 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:49.615084 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:49.615198 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 03:12:49.615249 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260812 03:12:49.615589 18681 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 71ef0c4e88d543d4bf8527c105a69114: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:12:49.615700 18681 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:12:49.616165 18682 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:12:49.616961 18682 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:12:49.622280 18682 catalog_manager.cc:1347] Generated new cluster ID: c3a44f8fdcd24087b49795fa5352e2b6
I20260812 03:12:49.622382 18682 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:12:49.643318 18682 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:12:49.644680 18682 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:12:49.661335 18682 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71ef0c4e88d543d4bf8527c105a69114: Generated new TSK 0
I20260812 03:12:49.662324 18682 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:12:49.668942 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 18582
2026-08-12T03:12:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1510 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:12:49 dist-test-slave-zpfg krb5kdc[18688](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:12:49 dist-test-slave-zpfg krb5kdc[18688](info): set up 2 sockets
Aug 12 03:12:49 dist-test-slave-zpfg krb5kdc[18688](info): commencing operation
krb5kdc: starting...
W20260812 03:12:50.839179 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:12:50 dist-test-slave-zpfg krb5kdc[18688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504370, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:12:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:12:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:12:50.982112 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:45071
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:37473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:45071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:51.104360 18701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:51.104656 18701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:51.104699 18701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:51.108932 18701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:51.109020 18701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:51.109040 18701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:51.109059 18701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:51.109077 18701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:51.115094 18701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:37473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:45071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:45071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:51.116344 18701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:51.117553 18701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:51.127315 18708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:51.127415 18701 server_base.cc:1034] running on GCE node
W20260812 03:12:51.127444 18709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:51.127492 18711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:51.128248 18701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:51.129556 18701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:51.130755 18701 hybrid_clock.cc:648] HybridClock initialized: now 1786504371130725 us; error 44 us; skew 500 ppm
Aug 12 03:12:51 dist-test-slave-zpfg krb5kdc[18688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504371, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:51.133736 18701 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:12:51.134804 18701 webserver.cc:460] Webserver started at http://127.26.250.190:45819/ using document root <none> and password file <none>
I20260812 03:12:51.135399 18701 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:51.135462 18701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:51.135637 18701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:51.137844 18701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "100103d6d7a7424fb7e3a7159e150817"
format_stamp: "Formatted at 2026-08-12 03:12:51 on dist-test-slave-zpfg"
server_key: "504f4fc5d5ce3f590884190ae67bca9e"
server_key_iv: "e5673af8f8aaca3178501c2d7a468e03"
server_key_version: "encryptionkey@0"
I20260812 03:12:51.138505 18701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "100103d6d7a7424fb7e3a7159e150817"
format_stamp: "Formatted at 2026-08-12 03:12:51 on dist-test-slave-zpfg"
server_key: "504f4fc5d5ce3f590884190ae67bca9e"
server_key_iv: "e5673af8f8aaca3178501c2d7a468e03"
server_key_version: "encryptionkey@0"
I20260812 03:12:51.142788 18701 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:12:51.145542 18718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:51.147145 18701 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:12:51.147421 18701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "100103d6d7a7424fb7e3a7159e150817"
format_stamp: "Formatted at 2026-08-12 03:12:51 on dist-test-slave-zpfg"
server_key: "504f4fc5d5ce3f590884190ae67bca9e"
server_key_iv: "e5673af8f8aaca3178501c2d7a468e03"
server_key_version: "encryptionkey@0"
I20260812 03:12:51.147569 18701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:51.152799 18701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:51.156651 18701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:51.156898 18701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:51.164750 18701 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:45071
I20260812 03:12:51.164887 18773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:45071 every 8 connection(s)
I20260812 03:12:51.165822 18701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:12:51.169250 18774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:51.170539 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 18701
I20260812 03:12:51.170701 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:12:51.171001 27626 external_mini_cluster.cc:1428] Setting key 7a6565efffe4157322ae3320cc51e0b4
I20260812 03:12:51.177577 18774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 100103d6d7a7424fb7e3a7159e150817: Bootstrap starting.
I20260812 03:12:51.179081 18777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:51.172458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50843 (local address 127.26.250.190:45071)
0812 03:12:51.173374 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:51.173395 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:51.173443 (+ 48us) server_negotiation.cc:407] Connection header received
0812 03:12:51.174320 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:51.174375 (+ 55us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:51.174776 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:51.175172 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:51.176246 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:51.176254 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:51.176279 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:51.176293 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:51.176418 (+ 125us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:51.176451 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:51.176840 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:51.177030 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 03:12:51.177625 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":147,"threads_started":1}
I20260812 03:12:51.180764 18774 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 100103d6d7a7424fb7e3a7159e150817: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:51.181627 18774 log.cc:824] T 00000000000000000000000000000000 P 100103d6d7a7424fb7e3a7159e150817: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:51.184178 18774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 100103d6d7a7424fb7e3a7159e150817: No bootstrap required, opened a new log
I20260812 03:12:51.186379 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:12:51.188848 18774 raft_consensus.cc:348] T 00000000000000000000000000000000 P 100103d6d7a7424fb7e3a7159e150817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "100103d6d7a7424fb7e3a7159e150817" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45071 } }
I20260812 03:12:51.189195 18774 raft_consensus.cc:374] T 00000000000000000000000000000000 P 100103d6d7a7424fb7e3a7159e150817 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 03:12:51.189266 18774 raft_consensus.cc:729] T 00000000000000000000000000000000 P 100103d6d7a7424fb7e3a7159e150817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 100103d6d7a7424fb7e3a7159e150817, State: Initialized, Role: FOLLOWER
I20260812 03:12:51.189797 18774 consensus_queue.cc:260] T 00000000000000000000000000000000 P 100103d6d7a7424fb7e3a7159e150817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "100103d6d7a7424fb7e3a7159e150817" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45071 } }
I20260812 03:12:51.189983 18774 raft_consensus.cc:388] T 00000000000000000000000000000000 P 100103d6d7a7424fb7e3a7159e150817 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 03:12:51.190057 18774 raft_consensus.cc:482] T 00000000000000000000000000000000 P 100103d6d7a7424fb7e3a7159e150817 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 03:12:51.190166 18774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 100103d6d7a7424fb7e3a7159e150817 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:51.190613 18777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:51.188597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50917 (local address 127.26.250.190:45071)
0812 03:12:51.188746 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:51.188752 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:51.188900 (+ 148us) server_negotiation.cc:407] Connection header received
0812 03:12:51.189017 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:51.189020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:51.189061 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:51.189147 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:51.190119 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:51.190122 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:51.190125 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 03:12:51.190137 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:51.190168 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 03:12:51.190183 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:51.190298 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:51.190424 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 03:12:51.190472 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 03:12:51.191342 18774 raft_consensus.cc:504] T 00000000000000000000000000000000 P 100103d6d7a7424fb7e3a7159e150817 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "100103d6d7a7424fb7e3a7159e150817" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45071 } }
I20260812 03:12:51.191725 18774 leader_election.cc:302] T 00000000000000000000000000000000 P 100103d6d7a7424fb7e3a7159e150817 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 100103d6d7a7424fb7e3a7159e150817; no voters:
I20260812 03:12:51.192060 18774 leader_election.cc:288] T 00000000000000000000000000000000 P 100103d6d7a7424fb7e3a7159e150817 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 03:12:51.192169 18784 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 100103d6d7a7424fb7e3a7159e150817 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:51.192454 18784 raft_consensus.cc:686] T 00000000000000000000000000000000 P 100103d6d7a7424fb7e3a7159e150817 [term 1 LEADER]: Becoming Leader. State: Replica: 100103d6d7a7424fb7e3a7159e150817, State: Running, Role: LEADER
I20260812 03:12:51.192793 18784 consensus_queue.cc:237] T 00000000000000000000000000000000 P 100103d6d7a7424fb7e3a7159e150817 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "100103d6d7a7424fb7e3a7159e150817" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45071 } }
I20260812 03:12:51.193516 18774 sys_catalog.cc:564] T 00000000000000000000000000000000 P 100103d6d7a7424fb7e3a7159e150817 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:51.194514 18786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 100103d6d7a7424fb7e3a7159e150817 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 100103d6d7a7424fb7e3a7159e150817. Latest consensus state: current_term: 1 leader_uuid: "100103d6d7a7424fb7e3a7159e150817" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "100103d6d7a7424fb7e3a7159e150817" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45071 } } }
I20260812 03:12:51.194658 18786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 100103d6d7a7424fb7e3a7159e150817 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:51.194980 18785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 100103d6d7a7424fb7e3a7159e150817 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "100103d6d7a7424fb7e3a7159e150817" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "100103d6d7a7424fb7e3a7159e150817" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 45071 } } }
I20260812 03:12:51.195075 18785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 100103d6d7a7424fb7e3a7159e150817 [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:51.195248 18793 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:12:51.198918 18793 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:12:51.205253 18793 catalog_manager.cc:1347] Generated new cluster ID: c8eab9a33a0748f8bd040895978f63e0
I20260812 03:12:51.205335 18793 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:12:51.213618 18793 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:12:51.213970 18793 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:12:51.219097 18793 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 100103d6d7a7424fb7e3a7159e150817: Generated new TSK 0
I20260812 03:12:51.220110 18793 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 03:12:51.227833 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 18701
2026-08-12T03:12:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1557 ms)
[----------] 18 tests from AuthTokenIssuingTest (27903 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:12:51 dist-test-slave-zpfg krb5kdc[18806](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:12:51 dist-test-slave-zpfg krb5kdc[18806](info): set up 2 sockets
Aug 12 03:12:51 dist-test-slave-zpfg krb5kdc[18806](info): commencing operation
krb5kdc: starting...
W20260812 03:12:52.332870 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:12:52 dist-test-slave-zpfg krb5kdc[18806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:12:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:12:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:12:52.478444 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:41791
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:46151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:41791,127.26.250.189:42005,127.26.250.188:36233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:52.598628 18821 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:12:52.599004 18821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:52.599084 18821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:52.599182 18821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:52.603406 18821 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:52.603502 18821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:52.603541 18821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:52.603577 18821 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:52.603619 18821 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:52.609153 18821 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:46151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:41791,127.26.250.189:42005,127.26.250.188:36233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:41791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:52.610386 18821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:52.611472 18821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:52.620553 18826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:52.620553 18827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:52.620553 18829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:52.620893 18821 server_base.cc:1034] running on GCE node
I20260812 03:12:52.621652 18821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:52.622957 18821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:52.624161 18821 hybrid_clock.cc:648] HybridClock initialized: now 1786504372624143 us; error 36 us; skew 500 ppm
Aug 12 03:12:52 dist-test-slave-zpfg krb5kdc[18806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504372, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:52.627113 18821 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:12:52.628028 18821 webserver.cc:460] Webserver started at http://127.26.250.190:36113/ using document root <none> and password file <none>
I20260812 03:12:52.628563 18821 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:52.628623 18821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:52.628798 18821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:52.631000 18821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "70643bdd64a04c8e8d6b043d2b4ecddf"
format_stamp: "Formatted at 2026-08-12 03:12:52 on dist-test-slave-zpfg"
server_key: "068f4f83ee030399b792a46387e11608"
server_key_iv: "0be60ccf06e17271ec7691f2bb509c57"
server_key_version: "encryptionkey@0"
I20260812 03:12:52.631531 18821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "70643bdd64a04c8e8d6b043d2b4ecddf"
format_stamp: "Formatted at 2026-08-12 03:12:52 on dist-test-slave-zpfg"
server_key: "068f4f83ee030399b792a46387e11608"
server_key_iv: "0be60ccf06e17271ec7691f2bb509c57"
server_key_version: "encryptionkey@0"
I20260812 03:12:52.635510 18821 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:12:52.638082 18837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:52.639489 18821 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 03:12:52.639640 18821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "70643bdd64a04c8e8d6b043d2b4ecddf"
format_stamp: "Formatted at 2026-08-12 03:12:52 on dist-test-slave-zpfg"
server_key: "068f4f83ee030399b792a46387e11608"
server_key_iv: "0be60ccf06e17271ec7691f2bb509c57"
server_key_version: "encryptionkey@0"
I20260812 03:12:52.639758 18821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:52.654318 18821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:52.657995 18821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:52.658228 18821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:52.666117 18821 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:41791
I20260812 03:12:52.666515 18891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:41791 every 8 connection(s)
I20260812 03:12:52.667240 18821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:12:52.670271 18893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:52.674365 18893 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41791 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:12:52.676396 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 18821
I20260812 03:12:52.676548 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:12:52.676854 27626 external_mini_cluster.cc:1428] Setting key 2ca565a9c42929b39db88e49adcb3c22
Aug 12 03:12:52 dist-test-slave-zpfg krb5kdc[18806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504372, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.250.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.189@KRBTEST.COM" created.
I20260812 03:12:52.690927 18896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:52.677069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37640 (local address 127.26.250.190:41791)
0812 03:12:52.677566 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:52.677587 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:52.677884 (+ 297us) server_negotiation.cc:407] Connection header received
0812 03:12:52.679039 (+ 1155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:52.679043 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:52.679221 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:52.679424 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:52.680301 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:52.681194 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:52.682331 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:52.682801 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:52.682979 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:52.685394 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:52.685407 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:52.685418 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:52.685439 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:52.687269 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:52.687891 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:52.687898 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:52.687900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:52.688016 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:52.688358 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:52.688363 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:52.688365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:52.688816 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:52.688966 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:52.689355 (+ 389us) server_negotiation.cc:299] Negotiation successful
0812 03:12:52.689670 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":335,"thread_start_us":167,"threads_started":1}
I20260812 03:12:52.690927 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:52.677070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:41791 (local address 127.0.0.1:37640)
0812 03:12:52.677724 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:52.677764 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:52.678722 (+ 958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:52.679416 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:52.679421 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:52.679884 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:52.680129 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:52.680142 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:52.681333 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:52.681338 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:52.682178 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:52.682187 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:52.682966 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:52.682969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:52.683061 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:52.683690 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:52.683708 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:52.685235 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:52.687417 (+ 2182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:52.687426 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:52.687438 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:52.687762 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:52.688111 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:52.688116 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:52.688117 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:52.688254 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:52.688934 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:52.688940 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:52.689092 (+ 152us) client_negotiation.cc:769] Sending connection context
0812 03:12:52.689283 (+ 191us) client_negotiation.cc:240] Negotiation successful
0812 03:12:52.689735 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":119,"spinlock_wait_cycles":12160,"thread_start_us":200,"threads_started":1}
I20260812 03:12:52.694130 18893 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42005 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:12:52.695029 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:52.694718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:42005 (local address 127.0.0.1:33232)
0812 03:12:52.694883 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:52.694957 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.189:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 03:12:52.695227 18838 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.250.189:42005: connect: Connection refused (error 111)
I20260812 03:12:52.696153 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:52.695928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:42005 (local address 127.0.0.1:33248)
0812 03:12:52.696042 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:52.696086 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.189:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 03:12:52.696312 18893 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.189:42005: Network error: Client connection negotiation failed: client connection to 127.26.250.189:42005: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.189.keytab.
Entry for principal kudu/127.26.250.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.189.keytab.
I20260812 03:12:52.709507 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.189:42005
--webserver_interface=127.26.250.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.189.keytab
--principal=kudu/127.26.250.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:46151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:41791,127.26.250.189:42005,127.26.250.188:36233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:12:52.745574 18893 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42005 } attempt: 1
I20260812 03:12:52.746673 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:52.746271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:42005 (local address 127.0.0.1:33260)
0812 03:12:52.746499 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:52.746586 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.189:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260812 03:12:52.747501 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:52.747264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:42005 (local address 127.0.0.1:33266)
0812 03:12:52.747382 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:52.747438 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.189:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 03:12:52.747645 18893 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.189:42005: Network error: Client connection negotiation failed: client connection to 127.26.250.189:42005: connect: Connection refused (error 111)
I20260812 03:12:52.815845 18893 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42005 } attempt: 2
I20260812 03:12:52.816948 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:52.816533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:42005 (local address 127.0.0.1:33278)
0812 03:12:52.816746 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:52.816823 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.189:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260812 03:12:52.817763 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:52.817521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:42005 (local address 127.0.0.1:33294)
0812 03:12:52.817641 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:52.817701 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.189:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 03:12:52.817888 18893 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.189:42005: Network error: Client connection negotiation failed: client connection to 127.26.250.189:42005: connect: Connection refused (error 111)
W20260812 03:12:52.848210 18901 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:12:52.848609 18901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:52.848698 18901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:52.848793 18901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:52.853498 18901 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:52.853610 18901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:52.853663 18901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:52.853724 18901 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:52.853761 18901 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:52.859449 18901 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:46151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:41791,127.26.250.189:42005,127.26.250.188:36233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.189.keytab
--principal=kudu/127.26.250.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.189:42005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.250.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:52.860682 18901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:52.861776 18901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:52.871210 18910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:52.871198 18901 server_base.cc:1034] running on GCE node
W20260812 03:12:52.871167 18906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:52.871167 18907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:52.872087 18901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:52.873268 18901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:52.874429 18901 hybrid_clock.cc:648] HybridClock initialized: now 1786504372874412 us; error 28 us; skew 500 ppm
Aug 12 03:12:52 dist-test-slave-zpfg krb5kdc[18806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504372, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:52.877877 18901 init.cc:377] Logged in from keytab as kudu/127.26.250.189@KRBTEST.COM (short username kudu)
I20260812 03:12:52.878966 18901 webserver.cc:460] Webserver started at http://127.26.250.189:33773/ using document root <none> and password file <none>
I20260812 03:12:52.879573 18901 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:52.879670 18901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:52.879915 18901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:52.882179 18901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-1/data/instance:
uuid: "e0f570438f094c28bef5a55988feadaa"
format_stamp: "Formatted at 2026-08-12 03:12:52 on dist-test-slave-zpfg"
server_key: "448da07a496beadbb3c88e7d91322074"
server_key_iv: "e775441152b91288eb3dea690b431266"
server_key_version: "encryptionkey@0"
I20260812 03:12:52.882895 18901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-1/wal/instance:
uuid: "e0f570438f094c28bef5a55988feadaa"
format_stamp: "Formatted at 2026-08-12 03:12:52 on dist-test-slave-zpfg"
server_key: "448da07a496beadbb3c88e7d91322074"
server_key_iv: "e775441152b91288eb3dea690b431266"
server_key_version: "encryptionkey@0"
I20260812 03:12:52.886955 18901 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 03:12:52.889993 18917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:52.891517 18901 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:12:52.891701 18901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-1/wal
uuid: "e0f570438f094c28bef5a55988feadaa"
format_stamp: "Formatted at 2026-08-12 03:12:52 on dist-test-slave-zpfg"
server_key: "448da07a496beadbb3c88e7d91322074"
server_key_iv: "e775441152b91288eb3dea690b431266"
server_key_version: "encryptionkey@0"
I20260812 03:12:52.891841 18901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:52.909070 18893 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42005 } attempt: 3
I20260812 03:12:52.910117 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:52.909729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:42005 (local address 127.0.0.1:33304)
0812 03:12:52.909956 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:52.910030 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.189:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260812 03:12:52.911083 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:52.910839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:42005 (local address 127.0.0.1:33306)
0812 03:12:52.910965 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:52.911022 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.189:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260812 03:12:52.911242 18893 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.189:42005: Network error: Client connection negotiation failed: client connection to 127.26.250.189:42005: connect: Connection refused (error 111)
I20260812 03:12:52.913995 18901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:52.917795 18901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:52.918021 18901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:52.925741 18901 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.189:42005
I20260812 03:12:52.926105 18974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.189:42005 every 8 connection(s)
I20260812 03:12:52.926896 18901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-1/data/info.pb
I20260812 03:12:52.928506 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 18901
I20260812 03:12:52.928642 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-1/wal/instance
I20260812 03:12:52.928958 27626 external_mini_cluster.cc:1428] Setting key 6ea78a506341c0f199e2a457bb180a5e
I20260812 03:12:52.930513 18975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:52.934926 18975 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41791 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.250.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.188@KRBTEST.COM" created.
Aug 12 03:12:52 dist-test-slave-zpfg krb5kdc[18806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504372, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.189@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:12:52.951819 18896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:52.937029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37644 (local address 127.26.250.190:41791)
0812 03:12:52.937250 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:52.937255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:52.938229 (+ 974us) server_negotiation.cc:407] Connection header received
0812 03:12:52.939470 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:52.939473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:52.939523 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:52.939612 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:52.941045 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:52.941714 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:52.943715 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:52.944141 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:52.944288 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:52.946952 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:52.946956 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:52.946959 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:52.946997 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:52.949027 (+ 2030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:52.949606 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:52.949611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:52.949614 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:52.949761 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:52.950296 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:52.950299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:52.950301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:52.950598 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:52.950725 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:52.951508 (+ 783us) server_negotiation.cc:299] Negotiation successful
0812 03:12:52.951675 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260812 03:12:52.952757 18978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:52.937504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:41791 (local address 127.0.0.1:37644)
0812 03:12:52.938028 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:52.938086 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:52.939244 (+ 1158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:52.939784 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:52.939791 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:52.940225 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:52.940866 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:52.940882 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:52.941875 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:52.941881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:52.943512 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:52.943523 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:52.944321 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:52.944323 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:52.944574 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:52.945241 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:52.945258 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:52.946754 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:52.949182 (+ 2428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:52.949188 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:52.949201 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:52.949476 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:52.950005 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:52.950011 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:52.950016 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:52.950169 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:52.950742 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:52.950747 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:52.951052 (+ 305us) client_negotiation.cc:769] Sending connection context
0812 03:12:52.951378 (+ 326us) client_negotiation.cc:240] Negotiation successful
0812 03:12:52.951661 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":158,"threads_started":1}
I20260812 03:12:52.954283 18975 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42005 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:12:52 dist-test-slave-zpfg krb5kdc[18806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504372, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.189@KRBTEST.COM for kudu/127.26.250.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.188.keytab.
Entry for principal kudu/127.26.250.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.188.keytab.
I20260812 03:12:52.962590 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.188:36233
--webserver_interface=127.26.250.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.188.keytab
--principal=kudu/127.26.250.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:46151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:41791,127.26.250.189:42005,127.26.250.188:36233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:12:52.964489 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:52.955026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33314 (local address 127.26.250.189:42005)
0812 03:12:52.955389 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:52.955403 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:52.955426 (+ 23us) server_negotiation.cc:407] Connection header received
0812 03:12:52.955464 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:52.955468 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:52.955619 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:52.955862 (+ 243us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:52.956214 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:52.956999 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:52.957894 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:52.958277 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:52.958472 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:52.960581 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:52.960596 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:52.960608 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:52.960628 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:52.962505 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:52.963011 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:52.963016 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:52.963017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:52.963101 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:52.963421 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:52.963424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:52.963425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:52.963709 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:52.963869 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:52.964196 (+ 327us) server_negotiation.cc:299] Negotiation successful
0812 03:12:52.964332 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":150,"threads_started":1}
I20260812 03:12:52.964505 18978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:52.954926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:42005 (local address 127.0.0.1:33314)
0812 03:12:52.955028 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:52.955043 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:52.955124 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:52.955817 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:52.955820 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:52.956002 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:52.956092 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:52.956097 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:52.957123 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:52.957129 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:52.957780 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:52.957791 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:52.958444 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:52.958447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:52.958547 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:52.959078 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:52.959093 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:52.960443 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:52.962627 (+ 2184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:52.962631 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:52.962633 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:52.962903 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:52.963195 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:52.963201 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:52.963206 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:52.963310 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:52.963816 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:52.963822 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:52.963940 (+ 118us) client_negotiation.cc:769] Sending connection context
0812 03:12:52.964062 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 03:12:52.964226 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 03:12:52.966861 18975 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:12:52.967803 18978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:52.967427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:36233 (local address 127.0.0.1:43262)
0812 03:12:52.967585 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:52.967699 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:36233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260812 03:12:52.968046 18921 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.250.188:36233: connect: Connection refused (error 111)
I20260812 03:12:52.969061 18978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:52.968751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:36233 (local address 127.0.0.1:43276)
0812 03:12:52.968908 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:52.968975 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:36233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260812 03:12:52.969283 18975 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.188:36233: Network error: Client connection negotiation failed: client connection to 127.26.250.188:36233: connect: Connection refused (error 111)
I20260812 03:12:53.018601 18975 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 } attempt: 1
I20260812 03:12:53.019735 18978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:53.019345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:36233 (local address 127.0.0.1:43286)
0812 03:12:53.019527 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:53.019602 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:36233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 03:12:53.020551 18978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:53.020305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:36233 (local address 127.0.0.1:43292)
0812 03:12:53.020424 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:53.020483 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:36233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 03:12:53.020709 18975 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.188:36233: Network error: Client connection negotiation failed: client connection to 127.26.250.188:36233: connect: Connection refused (error 111)
I20260812 03:12:53.054430 18893 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42005 } attempt: 4
Aug 12 03:12:53 dist-test-slave-zpfg krb5kdc[18806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504372, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for kudu/127.26.250.189@KRBTEST.COM
I20260812 03:12:53.064502 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:53.055124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:42005 (local address 127.0.0.1:33326)
0812 03:12:53.055439 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:53.055452 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:53.055594 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:53.055912 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:53.055915 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:53.056113 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:53.056222 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:53.056228 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.057448 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.057453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:53.058009 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:53.058017 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.058600 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.058604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:53.058721 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:53.059354 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:53.059367 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:53.060682 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:53.062655 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:53.062659 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:53.062661 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:53.062943 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:53.063396 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:53.063399 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:53.063401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:53.063501 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:53.063921 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:53.063924 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:53.064040 (+ 116us) client_negotiation.cc:769] Sending connection context
0812 03:12:53.064155 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 03:12:53.064312 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212}
I20260812 03:12:53.064531 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:53.055195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33326 (local address 127.26.250.189:42005)
0812 03:12:53.055371 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:53.055377 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:53.055536 (+ 159us) server_negotiation.cc:407] Connection header received
0812 03:12:53.055727 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:53.055730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:53.055788 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:53.055881 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:53.056359 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.057313 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.058136 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.058472 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.058590 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:53.060851 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:53.060854 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:53.060856 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:53.060880 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:53.062484 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:53.063084 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:53.063087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:53.063089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:53.063283 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:53.063605 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:53.063607 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:53.063609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:53.063818 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:53.063909 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:53.064301 (+ 392us) server_negotiation.cc:299] Negotiation successful
0812 03:12:53.064413 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260812 03:12:53.065475 18893 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:12:53.066296 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:53.065987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:36233 (local address 127.0.0.1:43294)
0812 03:12:53.066142 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:53.066211 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:36233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 03:12:53.067134 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:53.066929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:36233 (local address 127.0.0.1:43298)
0812 03:12:53.067031 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:53.067083 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:36233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 03:12:53.067306 18893 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.188:36233: Network error: Client connection negotiation failed: client connection to 127.26.250.188:36233: connect: Connection refused (error 111)
I20260812 03:12:53.088876 18975 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 } attempt: 2
I20260812 03:12:53.089787 18978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:53.089439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:36233 (local address 127.0.0.1:43308)
0812 03:12:53.089608 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:53.089683 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:36233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 03:12:53.090924 18978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:53.090691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:36233 (local address 127.0.0.1:43322)
0812 03:12:53.090806 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:53.090861 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:36233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 03:12:53.091095 18975 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.188:36233: Network error: Client connection negotiation failed: client connection to 127.26.250.188:36233: connect: Connection refused (error 111)
W20260812 03:12:53.091830 18981 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:12:53.092144 18981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:53.092199 18981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:53.092304 18981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:53.096673 18981 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:53.096743 18981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:53.096763 18981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:53.096781 18981 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:53.096797 18981 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:53.102572 18981 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:46151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:41791,127.26.250.189:42005,127.26.250.188:36233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.188.keytab
--principal=kudu/127.26.250.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.188:36233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.250.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:53.103782 18981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:53.104882 18981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:53.114398 18988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:53.114403 18990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:53.114399 18987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:53.114591 18981 server_base.cc:1034] running on GCE node
I20260812 03:12:53.115320 18981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:53.116542 18981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:53.117735 18981 hybrid_clock.cc:648] HybridClock initialized: now 1786504373117719 us; error 35 us; skew 500 ppm
Aug 12 03:12:53 dist-test-slave-zpfg krb5kdc[18806](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504373, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:53.120800 18981 init.cc:377] Logged in from keytab as kudu/127.26.250.188@KRBTEST.COM (short username kudu)
I20260812 03:12:53.121857 18981 webserver.cc:460] Webserver started at http://127.26.250.188:38571/ using document root <none> and password file <none>
I20260812 03:12:53.122511 18981 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:53.122608 18981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:53.122864 18981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:53.125080 18981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-2/data/instance:
uuid: "bea4913e97484bf58f31718abe554ae8"
format_stamp: "Formatted at 2026-08-12 03:12:53 on dist-test-slave-zpfg"
server_key: "b5b0a15b1ba968bf2ee12f0b93830533"
server_key_iv: "fdecac301196e50352956aff4406a77c"
server_key_version: "encryptionkey@0"
I20260812 03:12:53.125682 18981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-2/wal/instance:
uuid: "bea4913e97484bf58f31718abe554ae8"
format_stamp: "Formatted at 2026-08-12 03:12:53 on dist-test-slave-zpfg"
server_key: "b5b0a15b1ba968bf2ee12f0b93830533"
server_key_iv: "fdecac301196e50352956aff4406a77c"
server_key_version: "encryptionkey@0"
I20260812 03:12:53.126474 18893 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 } attempt: 1
I20260812 03:12:53.127430 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:53.127116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:36233 (local address 127.0.0.1:43328)
0812 03:12:53.127277 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:53.127336 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:36233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 03:12:53.128230 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:53.127995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:36233 (local address 127.0.0.1:43342)
0812 03:12:53.128119 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:53.128173 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:36233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 03:12:53.128427 18893 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.188:36233: Network error: Client connection negotiation failed: client connection to 127.26.250.188:36233: connect: Connection refused (error 111)
I20260812 03:12:53.130405 18981 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260812 03:12:53.133154 18997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:53.134635 18981 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 03:12:53.134790 18981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-2/wal
uuid: "bea4913e97484bf58f31718abe554ae8"
format_stamp: "Formatted at 2026-08-12 03:12:53 on dist-test-slave-zpfg"
server_key: "b5b0a15b1ba968bf2ee12f0b93830533"
server_key_iv: "fdecac301196e50352956aff4406a77c"
server_key_version: "encryptionkey@0"
I20260812 03:12:53.134914 18981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:53.154016 18981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:53.157699 18981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:53.157944 18981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:53.165699 18981 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.188:36233
I20260812 03:12:53.166008 19056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.188:36233 every 8 connection(s)
I20260812 03:12:53.166843 18981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-2/data/info.pb
I20260812 03:12:53.169891 19057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:53.171984 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 18981
I20260812 03:12:53.172143 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786504075277362-27626-0/minicluster-data/master-2/wal/instance
I20260812 03:12:53.172443 27626 external_mini_cluster.cc:1428] Setting key 9f9a8b713183429504cb0521b9a92f19
I20260812 03:12:53.174811 19057 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41791 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:12:53.182330 18975 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 } attempt: 3
Aug 12 03:12:53 dist-test-slave-zpfg krb5kdc[18806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504373, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.188@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
Aug 12 03:12:53 dist-test-slave-zpfg krb5kdc[18806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.188@KRBTEST.COM
Aug 12 03:12:53 dist-test-slave-zpfg krb5kdc[18806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504372, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.189@KRBTEST.COM for kudu/127.26.250.188@KRBTEST.COM
I20260812 03:12:53.190100 18896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:53.177268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37650 (local address 127.26.250.190:41791)
0812 03:12:53.177455 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:53.177460 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:53.177821 (+ 361us) server_negotiation.cc:407] Connection header received
0812 03:12:53.177950 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:53.177953 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:53.178008 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:53.178097 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:53.178928 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.179462 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.180793 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.181104 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.181217 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:53.184567 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:53.184570 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:53.184572 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:53.184597 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:53.186639 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:53.187284 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:53.187288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:53.187290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:53.187396 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:53.187945 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:53.187948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:53.187951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:53.188215 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:53.188317 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:53.188980 (+ 663us) server_negotiation.cc:299] Negotiation successful
0812 03:12:53.189113 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 03:12:53.191346 19061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:53.177218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:41791 (local address 127.0.0.1:37650)
0812 03:12:53.177665 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:53.177719 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:53.177849 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:53.178155 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:53.178160 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:53.178601 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:53.178774 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:53.178784 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.179607 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.179610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:53.180542 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:53.180551 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.181277 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.181283 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:53.181406 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:53.181894 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:53.181917 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:53.184386 (+ 2469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:53.186754 (+ 2368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:53.186763 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:53.186774 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:53.187107 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:53.187525 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:53.187529 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:53.187532 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:53.187821 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:53.188394 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:53.188402 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:53.188683 (+ 281us) client_negotiation.cc:769] Sending connection context
0812 03:12:53.190252 (+ 1569us) client_negotiation.cc:240] Negotiation successful
0812 03:12:53.190551 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":111,"thread_start_us":196,"threads_started":1}
I20260812 03:12:53.191412 19060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:53.174089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43346 (local address 127.26.250.188:36233)
0812 03:12:53.174601 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:53.174623 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:53.174663 (+ 40us) server_negotiation.cc:407] Connection header received
0812 03:12:53.175576 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:53.175598 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:53.176105 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:53.176642 (+ 537us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:53.177596 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.178913 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.180046 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.180878 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.181035 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:53.184492 (+ 3457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:53.184499 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:53.184515 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:53.184540 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:53.186732 (+ 2192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:53.187576 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:53.187582 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:53.187584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:53.187725 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:53.188113 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:53.188118 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:53.188123 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:53.188677 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:53.188891 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:53.189821 (+ 930us) server_negotiation.cc:299] Negotiation successful
0812 03:12:53.190214 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":362,"thread_start_us":169,"threads_started":1}
I20260812 03:12:53.193444 19057 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42005 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:12:53.194968 18978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:53.182969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:36233 (local address 127.0.0.1:43360)
0812 03:12:53.183112 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:53.183128 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:53.183346 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:53.183656 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:53.183660 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:53.183839 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:53.183958 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:53.183965 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.185295 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.185301 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:53.186064 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:53.186080 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.186850 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.186856 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:53.186952 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:53.187601 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:53.187617 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:53.189284 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:53.192907 (+ 3623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:53.192910 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:53.192913 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:53.193188 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:53.193512 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:53.193516 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:53.193518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:53.193633 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:53.194450 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:53.194453 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:53.194578 (+ 125us) client_negotiation.cc:769] Sending connection context
0812 03:12:53.194668 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 03:12:53.194832 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 03:12:53.195575 19062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:53.183033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43360 (local address 127.26.250.188:36233)
0812 03:12:53.183415 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:53.183426 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:53.183440 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:12:53.183475 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:53.183478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:53.183520 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:53.183626 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:53.184075 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.184731 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.186207 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.186723 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.186837 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:53.189443 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:53.189447 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:53.189449 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:53.189473 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:53.192753 (+ 3280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:53.193317 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:53.193320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:53.193322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:53.193415 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:53.193750 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:53.193754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:53.193755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:53.194016 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:53.195182 (+ 1166us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:53.195256 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 03:12:53.195386 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":126,"threads_started":1}
I20260812 03:12:53.195583 18893 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 } attempt: 2
I20260812 03:12:53.199635 18975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0f570438f094c28bef5a55988feadaa: Bootstrap starting.
Aug 12 03:12:53 dist-test-slave-zpfg krb5kdc[18806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504373, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.188@KRBTEST.COM for kudu/127.26.250.189@KRBTEST.COM
Aug 12 03:12:53 dist-test-slave-zpfg krb5kdc[18806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
Aug 12 03:12:53 dist-test-slave-zpfg krb5kdc[18806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504372, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for kudu/127.26.250.188@KRBTEST.COM
I20260812 03:12:53.202849 18975 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0f570438f094c28bef5a55988feadaa: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:53.203923 18975 log.cc:824] T 00000000000000000000000000000000 P e0f570438f094c28bef5a55988feadaa: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:53.204895 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:53.194066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33330 (local address 127.26.250.189:42005)
0812 03:12:53.194184 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:53.194189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:53.194234 (+ 45us) server_negotiation.cc:407] Connection header received
0812 03:12:53.194408 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:53.194411 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:53.194457 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:53.194533 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:53.195343 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.196102 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.197287 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.197619 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.197708 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:53.201003 (+ 3295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:53.201007 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:53.201010 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:53.201037 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:53.202829 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:53.203380 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:53.203384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:53.203386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:53.203479 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:53.203809 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:53.203812 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:53.203814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:53.204123 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:53.204206 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:53.204571 (+ 365us) server_negotiation.cc:299] Negotiation successful
0812 03:12:53.204714 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 03:12:53.205343 18896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:53.194442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37652 (local address 127.26.250.190:41791)
0812 03:12:53.194551 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:53.194555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:53.194567 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:12:53.194604 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:53.194607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:53.194654 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:53.194732 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:53.195826 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.196539 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.197617 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.197914 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.197987 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:53.201108 (+ 3121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:53.201112 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:53.201115 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:53.201140 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:53.203174 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:53.203610 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:53.203614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:53.203615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:53.203691 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:53.204027 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:53.204032 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:53.204033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:53.204276 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:53.204485 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:53.204895 (+ 410us) server_negotiation.cc:299] Negotiation successful
0812 03:12:53.205203 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:12:53.205870 19061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:53.193994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:42005 (local address 127.0.0.1:33330)
0812 03:12:53.194179 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:53.194194 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:53.194274 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:53.194662 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:53.194664 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:53.194874 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:53.195016 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:53.195024 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.196255 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.196257 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:53.197151 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:53.197160 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.197855 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.197858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:53.197952 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:53.198687 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:53.198701 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:53.200836 (+ 2135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:53.202963 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:53.202968 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:53.202971 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:53.203301 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:53.203586 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:53.203590 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:53.203593 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:53.203696 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:53.204263 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:53.204267 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:53.204366 (+ 99us) client_negotiation.cc:769] Sending connection context
0812 03:12:53.205486 (+ 1120us) client_negotiation.cc:240] Negotiation successful
0812 03:12:53.205672 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260812 03:12:53.206718 18975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0f570438f094c28bef5a55988feadaa: No bootstrap required, opened a new log
I20260812 03:12:53.206805 19057 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:12:53.207181 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:53.195971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:36233 (local address 127.0.0.1:43370)
0812 03:12:53.196310 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:53.196325 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:53.196408 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:53.196856 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:53.196859 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:53.197047 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:53.197134 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:53.197140 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.198179 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.198183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:53.199035 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:53.199045 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.199714 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.199718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:53.199802 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:53.200228 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:53.200248 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:53.202275 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:53.204795 (+ 2520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:53.204798 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:53.204800 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:53.205085 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:53.206104 (+ 1019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:53.206108 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:53.206110 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:53.206204 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:53.206708 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:53.206711 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:53.206813 (+ 102us) client_negotiation.cc:769] Sending connection context
0812 03:12:53.206909 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 03:12:53.207037 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265}
I20260812 03:12:53.207748 19060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:53.196529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43370 (local address 127.26.250.188:36233)
0812 03:12:53.196639 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:53.196644 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:53.196657 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:12:53.196689 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:53.196692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:53.196730 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:53.197274 (+ 544us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:53.197354 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.198030 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.199210 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.199569 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.200014 (+ 445us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:53.202457 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:53.202460 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:53.202463 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:53.202485 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:53.204667 (+ 2182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:53.205910 (+ 1243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:53.205913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:53.205914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:53.205995 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:53.206322 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:53.206325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:53.206326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:53.206586 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:53.207434 (+ 848us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:53.207508 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 03:12:53.207618 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 03:12:53.210991 18893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf: Bootstrap starting.
Aug 12 03:12:53 dist-test-slave-zpfg krb5kdc[18806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504373, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.188@KRBTEST.COM for kudu/127.26.250.188@KRBTEST.COM
I20260812 03:12:53.213907 18893 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:53.211457 18975 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0f570438f094c28bef5a55988feadaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41791 } } peers { permanent_uuid: "e0f570438f094c28bef5a55988feadaa" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42005 } } peers { permanent_uuid: "bea4913e97484bf58f31718abe554ae8" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 } }
I20260812 03:12:53.214167 18975 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0f570438f094c28bef5a55988feadaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0f570438f094c28bef5a55988feadaa, State: Initialized, Role: FOLLOWER
Aug 12 03:12:53 dist-test-slave-zpfg krb5kdc[18806](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504372, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.189@KRBTEST.COM
I20260812 03:12:53.214568 18975 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0f570438f094c28bef5a55988feadaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41791 } } peers { permanent_uuid: "e0f570438f094c28bef5a55988feadaa" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42005 } } peers { permanent_uuid: "bea4913e97484bf58f31718abe554ae8" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 } }
I20260812 03:12:53.215175 18893 log.cc:824] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:53.216176 18975 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0f570438f094c28bef5a55988feadaa [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:53.218632 19064 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0f570438f094c28bef5a55988feadaa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41791 } } peers { permanent_uuid: "e0f570438f094c28bef5a55988feadaa" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42005 } } peers { permanent_uuid: "bea4913e97484bf58f31718abe554ae8" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 } } }
I20260812 03:12:53.218851 19064 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0f570438f094c28bef5a55988feadaa [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:12:53.218854 19061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:53.207271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:36233 (local address 127.0.0.1:43382)
0812 03:12:53.207648 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:53.207661 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:53.207763 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:53.208073 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:53.208075 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:53.208404 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:53.208528 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:53.208535 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.209644 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.209647 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:53.210718 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:53.210730 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.211534 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.211537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:53.211636 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:53.212435 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:53.212452 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:53.214977 (+ 2525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:53.217190 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:53.217192 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:53.217195 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:53.217431 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:53.217672 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:53.217674 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:53.217676 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:53.217739 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:53.218256 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:53.218259 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:53.218412 (+ 153us) client_negotiation.cc:769] Sending connection context
0812 03:12:53.218523 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 03:12:53.218671 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335}
I20260812 03:12:53.219466 19062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:53.207489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43382 (local address 127.26.250.188:36233)
0812 03:12:53.207592 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:53.207596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:53.207709 (+ 113us) server_negotiation.cc:407] Connection header received
0812 03:12:53.207851 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:53.207856 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:53.207895 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:53.207947 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:53.208654 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.209513 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.210867 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.211271 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.211346 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:53.215137 (+ 3791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:53.215141 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:53.215144 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:53.215166 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:53.217066 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:53.217520 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:53.217523 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:53.217525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:53.217587 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:53.217856 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:53.217863 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:53.217868 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:53.218151 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:53.218263 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:53.219207 (+ 944us) server_negotiation.cc:299] Negotiation successful
0812 03:12:53.219320 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260812 03:12:53.221704 19076 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e0f570438f094c28bef5a55988feadaa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:12:53.221822 19076 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:12:53.222689 19057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bea4913e97484bf58f31718abe554ae8: Bootstrap starting.
I20260812 03:12:53.220836 18901 master_runner.cc:186] Error getting master registration for 127.26.250.190:41791: OK, instance_id { permanent_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" instance_seqno: 1786504372663810 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:12:53.225044 18901 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:12:53.225525 18893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf: No bootstrap required, opened a new log
I20260812 03:12:53.225884 19057 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bea4913e97484bf58f31718abe554ae8: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:53.226900 19057 log.cc:824] T 00000000000000000000000000000000 P bea4913e97484bf58f31718abe554ae8: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:53.230089 18893 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41791 } } peers { permanent_uuid: "e0f570438f094c28bef5a55988feadaa" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42005 } } peers { permanent_uuid: "bea4913e97484bf58f31718abe554ae8" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 } }
I20260812 03:12:53.230381 18893 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70643bdd64a04c8e8d6b043d2b4ecddf, State: Initialized, Role: FOLLOWER
I20260812 03:12:53.230840 18893 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41791 } } peers { permanent_uuid: "e0f570438f094c28bef5a55988feadaa" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42005 } } peers { permanent_uuid: "bea4913e97484bf58f31718abe554ae8" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 } }
I20260812 03:12:53.231501 19079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41791 } } peers { permanent_uuid: "e0f570438f094c28bef5a55988feadaa" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42005 } } peers { permanent_uuid: "bea4913e97484bf58f31718abe554ae8" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 } } }
I20260812 03:12:53.231696 19079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:12:53.232499 18893 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:53.234166 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:53.207126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33342 (local address 127.26.250.189:42005)
0812 03:12:53.207371 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:53.207376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:53.207554 (+ 178us) server_negotiation.cc:407] Connection header received
0812 03:12:53.207761 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:53.207764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:53.207816 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:53.207879 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:53.208794 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.209514 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.210880 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.211256 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.211346 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:53.223111 (+ 11765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:53.223116 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:53.223119 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:53.223151 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:53.231833 (+ 8682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:53.232418 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:53.232422 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:53.232424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:53.232522 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:53.232889 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:53.232893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:53.232895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:53.233108 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:53.233337 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:53.233938 (+ 601us) server_negotiation.cc:299] Negotiation successful
0812 03:12:53.234050 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168}
I20260812 03:12:53.237840 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 03:12:53.238111 19090 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:12:53.238196 19090 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:12:53.239416 18864 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37652
I20260812 03:12:53.239750 18864 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 03:12:53.239882 18864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:53.240594 19057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bea4913e97484bf58f31718abe554ae8: No bootstrap required, opened a new log
I20260812 03:12:53.241586 18864 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41791 } } peers { permanent_uuid: "e0f570438f094c28bef5a55988feadaa" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42005 } } peers { permanent_uuid: "bea4913e97484bf58f31718abe554ae8" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 } }
I20260812 03:12:53.242460 18864 leader_election.cc:288] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf [CANDIDATE]: Term 1 election: Requested vote from peers e0f570438f094c28bef5a55988feadaa (127.26.250.189:42005), bea4913e97484bf58f31718abe554ae8 (127.26.250.188:36233)
I20260812 03:12:53.245133 19057 raft_consensus.cc:348] T 00000000000000000000000000000000 P bea4913e97484bf58f31718abe554ae8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41791 } } peers { permanent_uuid: "e0f570438f094c28bef5a55988feadaa" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42005 } } peers { permanent_uuid: "bea4913e97484bf58f31718abe554ae8" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 } }
I20260812 03:12:53.245399 19057 raft_consensus.cc:729] T 00000000000000000000000000000000 P bea4913e97484bf58f31718abe554ae8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bea4913e97484bf58f31718abe554ae8, State: Initialized, Role: FOLLOWER
I20260812 03:12:53.245721 19057 consensus_queue.cc:260] T 00000000000000000000000000000000 P bea4913e97484bf58f31718abe554ae8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41791 } } peers { permanent_uuid: "e0f570438f094c28bef5a55988feadaa" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42005 } } peers { permanent_uuid: "bea4913e97484bf58f31718abe554ae8" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 } }
I20260812 03:12:53.247061 18947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e0f570438f094c28bef5a55988feadaa"
I20260812 03:12:53.247155 19057 sys_catalog.cc:564] T 00000000000000000000000000000000 P bea4913e97484bf58f31718abe554ae8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:53.247334 18947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0f570438f094c28bef5a55988feadaa [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:53.248884 18947 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e0f570438f094c28bef5a55988feadaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70643bdd64a04c8e8d6b043d2b4ecddf in term 1.
I20260812 03:12:53.249221 19026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bea4913e97484bf58f31718abe554ae8"
I20260812 03:12:53.249341 18838 leader_election.cc:302] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70643bdd64a04c8e8d6b043d2b4ecddf, e0f570438f094c28bef5a55988feadaa; no voters:
I20260812 03:12:53.249459 19026 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bea4913e97484bf58f31718abe554ae8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:53.250465 19079 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:53.250712 19079 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf [term 1 LEADER]: Becoming Leader. State: Replica: 70643bdd64a04c8e8d6b043d2b4ecddf, State: Running, Role: LEADER
I20260812 03:12:53.251071 19079 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41791 } } peers { permanent_uuid: "e0f570438f094c28bef5a55988feadaa" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42005 } } peers { permanent_uuid: "bea4913e97484bf58f31718abe554ae8" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 } }
I20260812 03:12:53.253111 19079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70643bdd64a04c8e8d6b043d2b4ecddf. Latest consensus state: current_term: 1 leader_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41791 } } peers { permanent_uuid: "e0f570438f094c28bef5a55988feadaa" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42005 } } peers { permanent_uuid: "bea4913e97484bf58f31718abe554ae8" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 } } }
I20260812 03:12:53.253257 19079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:53.257052 18896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:53.246619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37664 (local address 127.26.250.190:41791)
0812 03:12:53.246759 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:53.246765 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:53.246781 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:12:53.246950 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:53.246954 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:53.247000 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:53.247055 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:53.248154 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.248906 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.250768 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.251090 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.251186 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:53.253175 (+ 1989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:53.253182 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:53.253185 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:53.253212 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:53.254999 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:53.255570 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:53.255574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:53.255576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:53.255665 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:53.256024 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:53.256027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:53.256029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:53.256285 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:53.256386 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:53.256760 (+ 374us) server_negotiation.cc:299] Negotiation successful
0812 03:12:53.256898 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 03:12:53.258706 19026 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bea4913e97484bf58f31718abe554ae8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70643bdd64a04c8e8d6b043d2b4ecddf in term 1.
I20260812 03:12:53.243881 18821 master_runner.cc:186] Error getting master registration for 127.26.250.188:36233: OK, instance_id { permanent_uuid: "bea4913e97484bf58f31718abe554ae8" instance_seqno: 1786504373163631 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:12:53.260492 18821 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:12:53.264407 19115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:12:53.266285 19115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:12:53.269244 19098 sys_catalog.cc:455] T 00000000000000000000000000000000 P bea4913e97484bf58f31718abe554ae8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41791 } } peers { permanent_uuid: "e0f570438f094c28bef5a55988feadaa" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42005 } } peers { permanent_uuid: "bea4913e97484bf58f31718abe554ae8" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 } } }
I20260812 03:12:53.269541 19098 sys_catalog.cc:458] T 00000000000000000000000000000000 P bea4913e97484bf58f31718abe554ae8 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:12:53.269922 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:53.246112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33358 (local address 127.26.250.189:42005)
0812 03:12:53.246245 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:53.246250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:53.260209 (+ 13959us) server_negotiation.cc:407] Connection header received
0812 03:12:53.260427 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:53.260430 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:53.260481 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:53.260535 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:53.261527 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.262254 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.263393 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.263702 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.263780 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:53.265706 (+ 1926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:53.265710 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:53.265713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:53.265747 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:53.267604 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:53.268138 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:53.268142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:53.268144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:53.268424 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:53.268774 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:53.268777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:53.268779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:53.269040 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:53.269197 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:53.269591 (+ 394us) server_negotiation.cc:299] Negotiation successful
0812 03:12:53.269751 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260812 03:12:53.270097 19112 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bea4913e97484bf58f31718abe554ae8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:12:53.270201 19112 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:12:53.276280 19026 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bea4913e97484bf58f31718abe554ae8 [term 1 FOLLOWER]: Refusing update from remote peer 70643bdd64a04c8e8d6b043d2b4ecddf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:12:53.277232 18947 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e0f570438f094c28bef5a55988feadaa [term 1 FOLLOWER]: Refusing update from remote peer 70643bdd64a04c8e8d6b043d2b4ecddf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:12:53.278100 19079 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0f570438f094c28bef5a55988feadaa" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:12:53.279438 19079 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf [LEADER]: Connected to new peer: Peer: permanent_uuid: "bea4913e97484bf58f31718abe554ae8" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:12:53.288220 19064 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0f570438f094c28bef5a55988feadaa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70643bdd64a04c8e8d6b043d2b4ecddf. Latest consensus state: current_term: 1 leader_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41791 } } peers { permanent_uuid: "e0f570438f094c28bef5a55988feadaa" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42005 } } peers { permanent_uuid: "bea4913e97484bf58f31718abe554ae8" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 } } }
I20260812 03:12:53.288558 19064 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0f570438f094c28bef5a55988feadaa [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:12:53.290227 19098 sys_catalog.cc:455] T 00000000000000000000000000000000 P bea4913e97484bf58f31718abe554ae8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70643bdd64a04c8e8d6b043d2b4ecddf. Latest consensus state: current_term: 1 leader_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41791 } } peers { permanent_uuid: "e0f570438f094c28bef5a55988feadaa" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42005 } } peers { permanent_uuid: "bea4913e97484bf58f31718abe554ae8" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 } } }
I20260812 03:12:53.290408 19098 sys_catalog.cc:458] T 00000000000000000000000000000000 P bea4913e97484bf58f31718abe554ae8 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:12:53.296043 19062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:53.264900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43396 (local address 127.26.250.188:36233)
0812 03:12:53.265036 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:53.265043 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:53.269965 (+ 4922us) server_negotiation.cc:407] Connection header received
0812 03:12:53.270569 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:53.270573 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:53.270627 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:53.270717 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:53.271726 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.272249 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.273125 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:53.273368 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:53.273488 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:53.291684 (+ 18196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:53.291690 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:53.291693 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:53.291735 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:53.293704 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:53.294397 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:53.294401 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:53.294404 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:53.294501 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:53.294863 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:53.294867 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:53.294869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:53.295137 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:53.295262 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:53.295722 (+ 460us) server_negotiation.cc:299] Negotiation successful
0812 03:12:53.295864 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:12:53.301522 19079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41791 } } peers { permanent_uuid: "e0f570438f094c28bef5a55988feadaa" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42005 } } peers { permanent_uuid: "bea4913e97484bf58f31718abe554ae8" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 } } }
I20260812 03:12:53.301704 19079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:53.301860 19118 mvcc.cc:205] Tried to move back new op lower bound from 7317521912922128384 to 7317521912840441856. Current Snapshot: MvccSnapshot[applied={T|T < 7317521912922128384}]
I20260812 03:12:53.303562 19064 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0f570438f094c28bef5a55988feadaa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41791 } } peers { permanent_uuid: "e0f570438f094c28bef5a55988feadaa" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42005 } } peers { permanent_uuid: "bea4913e97484bf58f31718abe554ae8" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 } } }
I20260812 03:12:53.303717 19064 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0f570438f094c28bef5a55988feadaa [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:12:53.306483 19120 mvcc.cc:205] Tried to move back new op lower bound from 7317521912922128384 to 7317521912840441856. Current Snapshot: MvccSnapshot[applied={T|T < 7317521912922128384}]
I20260812 03:12:53.307883 19113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41791 } } peers { permanent_uuid: "e0f570438f094c28bef5a55988feadaa" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42005 } } peers { permanent_uuid: "bea4913e97484bf58f31718abe554ae8" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 } } }
I20260812 03:12:53.308045 19113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:53.309221 19098 sys_catalog.cc:455] T 00000000000000000000000000000000 P bea4913e97484bf58f31718abe554ae8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70643bdd64a04c8e8d6b043d2b4ecddf" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 41791 } } peers { permanent_uuid: "e0f570438f094c28bef5a55988feadaa" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 42005 } } peers { permanent_uuid: "bea4913e97484bf58f31718abe554ae8" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 36233 } } }
I20260812 03:12:53.309371 19098 sys_catalog.cc:458] T 00000000000000000000000000000000 P bea4913e97484bf58f31718abe554ae8 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:12:53.309669 19122 mvcc.cc:205] Tried to move back new op lower bound from 7317521912922128384 to 7317521912840441856. Current Snapshot: MvccSnapshot[applied={T|T < 7317521912922128384}]
I20260812 03:12:53.312705 19115 catalog_manager.cc:1347] Generated new cluster ID: 3e83b2395c6a43e3be28919565be36c7
I20260812 03:12:53.312798 19115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:12:53.341810 19115 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:12:53.343526 19115 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:12:53.365624 19115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70643bdd64a04c8e8d6b043d2b4ecddf: Generated new TSK 0
I20260812 03:12:53.366787 19115 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:12:53.407276 19062 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:12:53.404880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43412 (local address 127.26.250.188:36233)
0812 03:12:53.405039 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:53.405044 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:53.405860 (+ 816us) server_negotiation.cc:407] Connection header received
0812 03:12:53.406104 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:53.406108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:53.406183 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:53.406250 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:53.407118 (+ 868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43412: BlockingRecv error: recv got EOF from 127.0.0.1:43412 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260812 03:12:53.410192 18896 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:12:53.408298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37666 (local address 127.26.250.190:41791)
0812 03:12:53.408430 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:53.408436 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:53.409134 (+ 698us) server_negotiation.cc:407] Connection header received
0812 03:12:53.409333 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:53.409336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:53.409389 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:53.409449 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:53.410050 (+ 601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37666: BlockingRecv error: recv got EOF from 127.0.0.1:37666 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260812 03:12:53.411949 18980 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:12:53.408823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33364 (local address 127.26.250.189:42005)
0812 03:12:53.408969 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:53.408975 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:53.410890 (+ 1915us) server_negotiation.cc:407] Connection header received
0812 03:12:53.411092 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:53.411097 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:53.411164 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:53.411221 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:53.411809 (+ 588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33364: BlockingRecv error: recv got EOF from 127.0.0.1:33364 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260812 03:12:53.417007 18980 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:12:53.415131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33374 (local address 127.26.250.189:42005)
0812 03:12:53.415268 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:53.415273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:53.415819 (+ 546us) server_negotiation.cc:407] Connection header received
0812 03:12:53.416073 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:53.416077 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:53.416194 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:53.416260 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:53.416887 (+ 627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33374: BlockingRecv error: recv got EOF from 127.0.0.1:33374 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260812 03:12:53.424486 18980 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:12:53.422540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33380 (local address 127.26.250.189:42005)
0812 03:12:53.422689 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:53.422695 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:53.423283 (+ 588us) server_negotiation.cc:407] Connection header received
0812 03:12:53.423528 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:53.423532 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:53.423597 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:53.423664 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:53.424354 (+ 690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33380: BlockingRecv error: recv got EOF from 127.0.0.1:33380 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260812 03:12:53.431615 18980 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:12:53.429489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33390 (local address 127.26.250.189:42005)
0812 03:12:53.429619 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:53.429624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:53.430312 (+ 688us) server_negotiation.cc:407] Connection header received
0812 03:12:53.430702 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:53.430706 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:53.430770 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:53.430834 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:53.431487 (+ 653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33390: BlockingRecv error: recv got EOF from 127.0.0.1:33390 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260812 03:12:53.440900 18980 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:12:53.438849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33400 (local address 127.26.250.189:42005)
0812 03:12:53.439037 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:53.439042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:53.439658 (+ 616us) server_negotiation.cc:407] Connection header received
0812 03:12:53.439953 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:53.439957 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:53.440031 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:53.440090 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:53.440772 (+ 682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33400: BlockingRecv error: recv got EOF from 127.0.0.1:33400 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
W20260812 03:12:53.454062 18980 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:12:53.452052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33412 (local address 127.26.250.189:42005)
0812 03:12:53.452185 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:53.452192 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:53.452832 (+ 640us) server_negotiation.cc:407] Connection header received
0812 03:12:53.453062 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:53.453067 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:53.453129 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:53.453188 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:53.453893 (+ 705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33412: BlockingRecv error: recv got EOF from 127.0.0.1:33412 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260812 03:12:53.474574 18980 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:12:53.473036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33414 (local address 127.26.250.189:42005)
0812 03:12:53.473175 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:53.473180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:53.473449 (+ 269us) server_negotiation.cc:407] Connection header received
0812 03:12:53.473630 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:53.473633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:53.473690 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:53.473784 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:53.474438 (+ 654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33414: BlockingRecv error: recv got EOF from 127.0.0.1:33414 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260812 03:12:53.510299 18980 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:12:53.508879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33420 (local address 127.26.250.189:42005)
0812 03:12:53.509064 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:53.509071 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:53.509306 (+ 235us) server_negotiation.cc:407] Connection header received
0812 03:12:53.509506 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:53.509513 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:53.509564 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:53.509660 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:53.510167 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33420: BlockingRecv error: recv got EOF from 127.0.0.1:33420 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260812 03:12:53.578485 18980 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:12:53.576899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33428 (local address 127.26.250.189:42005)
0812 03:12:53.577098 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:53.577102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:53.577229 (+ 127us) server_negotiation.cc:407] Connection header received
0812 03:12:53.577560 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:53.577563 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:53.577618 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:53.577718 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:53.578291 (+ 573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33428: BlockingRecv error: recv got EOF from 127.0.0.1:33428 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260812 03:12:53.709960 18980 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:12:53.708785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33432 (local address 127.26.250.189:42005)
0812 03:12:53.708966 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:53.708970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:53.709051 (+ 81us) server_negotiation.cc:407] Connection header received
0812 03:12:53.709180 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:53.709182 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:53.709232 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:53.709309 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:53.709802 (+ 493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33432: BlockingRecv error: recv got EOF from 127.0.0.1:33432 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260812 03:12:53.969659 18980 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:12:53.968513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33438 (local address 127.26.250.189:42005)
0812 03:12:53.968677 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:53.968681 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:53.968800 (+ 119us) server_negotiation.cc:407] Connection header received
0812 03:12:53.968897 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:53.968900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:53.968948 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:53.969023 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:53.969513 (+ 490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33438: BlockingRecv error: recv got EOF from 127.0.0.1:33438 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 03:12:54.223317 19076 catalog_manager.cc:1259] Loaded cluster ID: 3e83b2395c6a43e3be28919565be36c7
I20260812 03:12:54.223409 19076 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e0f570438f094c28bef5a55988feadaa: loading cluster ID for follower catalog manager: success
I20260812 03:12:54.225201 19076 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e0f570438f094c28bef5a55988feadaa: acquiring CA information for follower catalog manager: success
I20260812 03:12:54.226388 19076 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e0f570438f094c28bef5a55988feadaa: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:12:54.271788 19112 catalog_manager.cc:1259] Loaded cluster ID: 3e83b2395c6a43e3be28919565be36c7
I20260812 03:12:54.271889 19112 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bea4913e97484bf58f31718abe554ae8: loading cluster ID for follower catalog manager: success
I20260812 03:12:54.273588 19112 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bea4913e97484bf58f31718abe554ae8: acquiring CA information for follower catalog manager: success
I20260812 03:12:54.274915 19112 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bea4913e97484bf58f31718abe554ae8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:12:54.488543 19193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:54.484133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33440 (local address 127.26.250.189:42005)
0812 03:12:54.484457 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:54.484461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:54.484479 (+ 18us) server_negotiation.cc:407] Connection header received
0812 03:12:54.484615 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:54.484617 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:54.484769 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:54.484882 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:12:54.485183 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:54.485820 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:54.486865 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:54.487102 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:54.487193 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:54.487598 (+ 405us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:12:54.488076 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:12:54.488179 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:54.488281 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 03:12:54.488341 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":107,"threads_started":1}
I20260812 03:12:54.496660 19201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:54.492377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43418 (local address 127.26.250.188:36233)
0812 03:12:54.492662 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:54.492666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:54.492681 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:12:54.492769 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:54.492772 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:54.492911 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:54.493020 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:12:54.493240 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:54.493819 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:54.494788 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:54.495065 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:54.495152 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:54.495563 (+ 411us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:12:54.496065 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:12:54.496194 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:54.496338 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 03:12:54.496440 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":84,"threads_started":1}
I20260812 03:12:54.505617 19207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:54.500547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37682 (local address 127.26.250.190:41791)
0812 03:12:54.501016 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:54.501021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:54.501041 (+ 20us) server_negotiation.cc:407] Connection header received
0812 03:12:54.501185 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:54.501189 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:54.501450 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:54.501668 (+ 218us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:12:54.501959 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:54.502796 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:54.503788 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:54.504012 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:54.504124 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:54.504582 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:12:54.504959 (+ 377us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:12:54.505052 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:54.505341 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 03:12:54.505406 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":104,"threads_started":1}
I20260812 03:12:54.505779 19193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:54.501201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33448 (local address 127.26.250.189:42005)
0812 03:12:54.501343 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:54.501348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:54.501385 (+ 37us) server_negotiation.cc:407] Connection header received
0812 03:12:54.501622 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:54.501628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:54.501705 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:54.501793 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:12:54.502252 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:54.503048 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:54.504488 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:54.504738 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:54.504868 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:54.505185 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:12:54.505272 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:12:54.505328 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:54.505520 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 03:12:54.505580 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 03:12:54.505856 19201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:54.500684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43426 (local address 127.26.250.188:36233)
0812 03:12:54.500840 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:54.500846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:54.501358 (+ 512us) server_negotiation.cc:407] Connection header received
0812 03:12:54.501566 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:54.501573 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:54.501645 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:54.501721 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:12:54.502276 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:54.502916 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:54.504455 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:54.504832 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:54.504934 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:54.505333 (+ 399us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:12:54.505415 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:12:54.505462 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:54.505631 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 03:12:54.505715 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 03:12:54.509895 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 18821
I20260812 03:12:54.520244 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 18901
I20260812 03:12:54.529603 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 18981
2026-08-12T03:12:54Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3303 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 03:12:54 dist-test-slave-zpfg krb5kdc[19213](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 03:12:54 dist-test-slave-zpfg krb5kdc[19213](info): set up 2 sockets
Aug 12 03:12:54 dist-test-slave-zpfg krb5kdc[19213](info): commencing operation
krb5kdc: starting...
W20260812 03:12:55.668615 27626 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/krb5kdc/test-user.keytab.
Aug 12 03:12:55 dist-test-slave-zpfg krb5kdc[19213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T03:12:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T03:12:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.26.250.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
Entry for principal kudu/127.26.250.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab.
I20260812 03:12:55.809945 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:43041
--webserver_interface=127.26.250.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:42527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:43041,127.26.250.189:41353,127.26.250.188:42319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 03:12:55.937335 19227 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:12:55.937687 19227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:55.937721 19227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:55.937811 19227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:55.942297 19227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:55.942411 19227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:55.942440 19227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:55.942461 19227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:55.942478 19227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:55.948305 19227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:42527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:43041,127.26.250.189:41353,127.26.250.188:42319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.190.keytab
--principal=kudu/127.26.250.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.190:43041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.26.250.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:55.949486 19227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:55.950568 19227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:55.959869 19234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:55.959852 19238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:55.960063 19227 server_base.cc:1034] running on GCE node
W20260812 03:12:55.959959 19235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:55.960873 19227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:55.961972 19227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:55.963143 19227 hybrid_clock.cc:648] HybridClock initialized: now 1786504375963105 us; error 54 us; skew 500 ppm
Aug 12 03:12:55 dist-test-slave-zpfg krb5kdc[19213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504375, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:55.976850 19227 init.cc:377] Logged in from keytab as kudu/127.26.250.190@KRBTEST.COM (short username kudu)
I20260812 03:12:55.978008 19227 webserver.cc:460] Webserver started at http://127.26.250.190:34215/ using document root <none> and password file <none>
I20260812 03:12:55.978724 19227 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:55.978830 19227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:55.979089 19227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:55.981261 19227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-0/data/instance:
uuid: "d033b584459d464f8265d1e3f6fb378f"
format_stamp: "Formatted at 2026-08-12 03:12:55 on dist-test-slave-zpfg"
server_key: "db3b3b1172426bdbcb34c5cf9cdf2b69"
server_key_iv: "f291774df92c058fc9c47e2bf1eacafc"
server_key_version: "encryptionkey@0"
I20260812 03:12:55.981871 19227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-0/wal/instance:
uuid: "d033b584459d464f8265d1e3f6fb378f"
format_stamp: "Formatted at 2026-08-12 03:12:55 on dist-test-slave-zpfg"
server_key: "db3b3b1172426bdbcb34c5cf9cdf2b69"
server_key_iv: "f291774df92c058fc9c47e2bf1eacafc"
server_key_version: "encryptionkey@0"
I20260812 03:12:55.986271 19227 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260812 03:12:55.989392 19245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:55.990850 19227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 03:12:55.991017 19227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-0/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-0/wal
uuid: "d033b584459d464f8265d1e3f6fb378f"
format_stamp: "Formatted at 2026-08-12 03:12:55 on dist-test-slave-zpfg"
server_key: "db3b3b1172426bdbcb34c5cf9cdf2b69"
server_key_iv: "f291774df92c058fc9c47e2bf1eacafc"
server_key_version: "encryptionkey@0"
I20260812 03:12:55.991151 19227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:56.010989 19227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:56.015341 19227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:56.015625 19227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:56.024861 19227 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.190:43041
I20260812 03:12:56.025211 19303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.190:43041 every 8 connection(s)
I20260812 03:12:56.026119 19227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-0/data/info.pb
I20260812 03:12:56.028887 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 19227
I20260812 03:12:56.029016 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-0/wal/instance
I20260812 03:12:56.029368 19304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:56.029531 27626 external_mini_cluster.cc:1428] Setting key f111113b586841f1e11eefe5b6f50143
I20260812 03:12:56.033596 19304 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43041 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.26.250.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.189@KRBTEST.COM" created.
Aug 12 03:12:56 dist-test-slave-zpfg krb5kdc[19213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504375, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:12:56.051306 19309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.036470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:43041 (local address 127.0.0.1:36588)
0812 03:12:56.037094 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.037133 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:56.038046 (+ 913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:56.038813 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:56.038819 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:56.039262 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:56.039522 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:56.039536 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.040777 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.040781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:56.041572 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:56.041609 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.042372 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.042377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:56.042564 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:56.043420 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:56.043437 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:56.045090 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:56.047816 (+ 2726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:56.047826 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:56.047840 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:56.048082 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:56.048462 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:56.048465 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:56.048469 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:56.048585 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:56.049305 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:56.049312 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:56.049513 (+ 201us) client_negotiation.cc:769] Sending connection context
0812 03:12:56.049698 (+ 185us) client_negotiation.cc:240] Negotiation successful
0812 03:12:56.050101 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":27,"spinlock_wait_cycles":3328,"thread_start_us":146,"threads_started":1}
I20260812 03:12:56.051306 19308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.036469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36588 (local address 127.26.250.190:43041)
0812 03:12:56.036965 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:56.036985 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:56.037253 (+ 268us) server_negotiation.cc:407] Connection header received
0812 03:12:56.038424 (+ 1171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:56.038429 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:56.038626 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:56.038815 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:56.039672 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.040651 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.041720 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.042183 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.042473 (+ 290us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:56.045344 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:56.045351 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:56.045363 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:56.045387 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:56.047665 (+ 2278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.048213 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.048226 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.048227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.048355 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.048684 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.048687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.048691 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.049177 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:56.049327 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:56.049759 (+ 432us) server_negotiation.cc:299] Negotiation successful
0812 03:12:56.050149 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":232,"server-negotiator.queue_time_us":326,"thread_start_us":166,"threads_started":1}
I20260812 03:12:56.054327 19304 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.189" port: 41353 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:12:56.055425 19309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.054956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:41353 (local address 127.0.0.1:51112)
0812 03:12:56.055158 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.055329 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.189:41353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 03:12:56.055637 19249 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.250.189:41353: connect: Connection refused (error 111)
I20260812 03:12:56.056645 19309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.056430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:41353 (local address 127.0.0.1:51118)
0812 03:12:56.056539 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.056587 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.189:41353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 03:12:56.056941 19304 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.189:41353: Network error: Client connection negotiation failed: client connection to 127.26.250.189:41353: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.189.keytab.
Entry for principal kudu/127.26.250.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.189.keytab.
I20260812 03:12:56.063077 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.189:41353
--webserver_interface=127.26.250.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.189.keytab
--principal=kudu/127.26.250.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:42527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:43041,127.26.250.189:41353,127.26.250.188:42319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:12:56.106240 19304 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.189" port: 41353 } attempt: 1
I20260812 03:12:56.107378 19309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.107005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:41353 (local address 127.0.0.1:51122)
0812 03:12:56.107178 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.107246 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.189:41353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 03:12:56.108197 19309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.107962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:41353 (local address 127.0.0.1:51128)
0812 03:12:56.108080 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.108137 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.189:41353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 03:12:56.108345 19304 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.189:41353: Network error: Client connection negotiation failed: client connection to 127.26.250.189:41353: connect: Connection refused (error 111)
I20260812 03:12:56.176537 19304 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.189" port: 41353 } attempt: 2
I20260812 03:12:56.177443 19309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.177119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:41353 (local address 127.0.0.1:51138)
0812 03:12:56.177278 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.177361 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.189:41353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 03:12:56.178254 19309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.178030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:41353 (local address 127.0.0.1:51148)
0812 03:12:56.178140 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.178195 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.189:41353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 03:12:56.178511 19304 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.189:41353: Network error: Client connection negotiation failed: client connection to 127.26.250.189:41353: connect: Connection refused (error 111)
W20260812 03:12:56.188143 19312 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:12:56.188458 19312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:56.188483 19312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:56.188565 19312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:56.192929 19312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:56.193008 19312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:56.193028 19312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:56.193046 19312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:56.193086 19312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:56.199177 19312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:42527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:43041,127.26.250.189:41353,127.26.250.188:42319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.189.keytab
--principal=kudu/127.26.250.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.189:41353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.26.250.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:56.200453 19312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:56.201990 19312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:56.211723 19317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:56.211736 19320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:56.211736 19312 server_base.cc:1034] running on GCE node
W20260812 03:12:56.212106 19318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:56.212853 19312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:56.214291 19312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:56.215524 19312 hybrid_clock.cc:648] HybridClock initialized: now 1786504376215510 us; error 33 us; skew 500 ppm
Aug 12 03:12:56 dist-test-slave-zpfg krb5kdc[19213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504376, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:56.219424 19312 init.cc:377] Logged in from keytab as kudu/127.26.250.189@KRBTEST.COM (short username kudu)
I20260812 03:12:56.220556 19312 webserver.cc:460] Webserver started at http://127.26.250.189:42731/ using document root <none> and password file <none>
I20260812 03:12:56.221252 19312 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:56.221364 19312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:56.221637 19312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:56.224103 19312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-1/data/instance:
uuid: "420e374d01ec44f9a527b444de342b8b"
format_stamp: "Formatted at 2026-08-12 03:12:56 on dist-test-slave-zpfg"
server_key: "7c2754504b37d529ca0df9091864494c"
server_key_iv: "ae451561701e206be079dcb107c50cb2"
server_key_version: "encryptionkey@0"
I20260812 03:12:56.224781 19312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-1/wal/instance:
uuid: "420e374d01ec44f9a527b444de342b8b"
format_stamp: "Formatted at 2026-08-12 03:12:56 on dist-test-slave-zpfg"
server_key: "7c2754504b37d529ca0df9091864494c"
server_key_iv: "ae451561701e206be079dcb107c50cb2"
server_key_version: "encryptionkey@0"
I20260812 03:12:56.229332 19312 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260812 03:12:56.232335 19328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:56.233773 19312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 03:12:56.233952 19312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-1/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-1/wal
uuid: "420e374d01ec44f9a527b444de342b8b"
format_stamp: "Formatted at 2026-08-12 03:12:56 on dist-test-slave-zpfg"
server_key: "7c2754504b37d529ca0df9091864494c"
server_key_iv: "ae451561701e206be079dcb107c50cb2"
server_key_version: "encryptionkey@0"
I20260812 03:12:56.234090 19312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:56.250293 19312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:56.254076 19312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:56.254446 19312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:56.263203 19312 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.189:41353
I20260812 03:12:56.263583 19385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.189:41353 every 8 connection(s)
I20260812 03:12:56.264451 19312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-1/data/info.pb
I20260812 03:12:56.267831 19386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:56.269742 19304 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.189" port: 41353 } attempt: 3
I20260812 03:12:56.271885 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 19312
I20260812 03:12:56.272054 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-1/wal/instance
I20260812 03:12:56.272517 27626 external_mini_cluster.cc:1428] Setting key 560d7e7a611dff03e027d323324e6366
I20260812 03:12:56.273144 19386 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43041 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:12:56 dist-test-slave-zpfg krb5kdc[19213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504375, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for kudu/127.26.250.189@KRBTEST.COM
Aug 12 03:12:56 dist-test-slave-zpfg krb5kdc[19213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504376, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.189@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.26.250.188@KRBTEST.COM; defaulting to no policy
I20260812 03:12:56.286088 19309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.270539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:41353 (local address 127.0.0.1:51164)
0812 03:12:56.270693 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.270713 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:56.270823 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:56.274499 (+ 3676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:56.274503 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:56.274729 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:56.274906 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:56.274914 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.276970 (+ 2056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.276975 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:56.277830 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:56.277842 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.278829 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.278833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:56.278950 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:56.279586 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:56.279612 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:56.281355 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:56.284002 (+ 2647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:56.284008 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:56.284010 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:56.284318 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:56.284724 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:56.284727 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:56.284729 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:56.284820 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:56.285511 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:56.285514 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:56.285630 (+ 116us) client_negotiation.cc:769] Sending connection context
0812 03:12:56.285741 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 03:12:56.285900 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.26.250.188@KRBTEST.COM" created.
I20260812 03:12:56.287420 19308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.275649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36596 (local address 127.26.250.190:43041)
0812 03:12:56.275794 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:56.275800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:56.276286 (+ 486us) server_negotiation.cc:407] Connection header received
0812 03:12:56.276573 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:56.276576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:56.276631 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:56.276725 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:56.277689 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.278591 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.279775 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.280033 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.280144 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:56.283057 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:56.283071 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:56.283073 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:56.283095 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:56.285306 (+ 2211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.285881 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.285884 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.285885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.285959 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.286326 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.286329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.286331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.286572 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:56.286670 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:56.287159 (+ 489us) server_negotiation.cc:299] Negotiation successful
0812 03:12:56.287289 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 03:12:56.287516 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.271516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51164 (local address 127.26.250.189:41353)
0812 03:12:56.272263 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:56.272289 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:56.272345 (+ 56us) server_negotiation.cc:407] Connection header received
0812 03:12:56.273489 (+ 1144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:56.273522 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:56.274079 (+ 557us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:56.274550 (+ 471us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:56.275135 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.276818 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.277954 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.278614 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.278851 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:56.281526 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:56.281536 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:56.281552 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:56.281582 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:56.283855 (+ 2273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.284455 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.284460 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.284466 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.284614 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.284916 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.284921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.284923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.285386 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:56.285531 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:56.285925 (+ 394us) server_negotiation.cc:299] Negotiation successful
0812 03:12:56.286255 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":200,"threads_started":1}
I20260812 03:12:56.287590 19390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.275642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:43041 (local address 127.0.0.1:36596)
0812 03:12:56.276145 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.276204 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:56.276468 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:56.276782 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:56.276789 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:56.277356 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:56.277524 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:56.277531 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.278742 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.278747 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:56.279644 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:56.279654 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.280208 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.280214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:56.280352 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:56.280978 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:56.281006 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:56.282898 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:56.285427 (+ 2529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:56.285439 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:56.285446 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:56.285717 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:56.286062 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:56.286065 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:56.286067 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:56.286190 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:56.286764 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:56.286769 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:56.286884 (+ 115us) client_negotiation.cc:769] Sending connection context
0812 03:12:56.287057 (+ 173us) client_negotiation.cc:240] Negotiation successful
0812 03:12:56.287284 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":206,"threads_started":1}
I20260812 03:12:56.289547 19386 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.189" port: 41353 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:12:56.289965 19304 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:12:56.290814 19309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.290461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:42319 (local address 127.0.0.1:54880)
0812 03:12:56.290644 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.290722 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:42319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260812 03:12:56.291788 19392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.291276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:42319 (local address 127.0.0.1:54890)
0812 03:12:56.291644 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.291702 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:42319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":129,"threads_started":1}
W20260812 03:12:56.291946 19304 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.188:42319: Network error: Client connection negotiation failed: client connection to 127.26.250.188:42319: connect: Connection refused (error 111)
Aug 12 03:12:56 dist-test-slave-zpfg krb5kdc[19213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504376, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.189@KRBTEST.COM for kudu/127.26.250.189@KRBTEST.COM
I20260812 03:12:56.299991 19390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.290148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:41353 (local address 127.0.0.1:51180)
0812 03:12:56.290367 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.290384 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:56.290464 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:56.290735 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:56.290738 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:56.290936 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:56.291049 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:56.291056 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.292145 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.292148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:56.292680 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:56.292688 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.293445 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.293449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:56.293547 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:56.294183 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:56.294204 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:56.295957 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:56.298137 (+ 2180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:56.298141 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:56.298143 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:56.298515 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:56.298854 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:56.298859 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:56.298861 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:56.298961 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:56.299389 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:56.299391 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:56.299510 (+ 119us) client_negotiation.cc:769] Sending connection context
0812 03:12:56.299615 (+ 105us) client_negotiation.cc:240] Negotiation successful
0812 03:12:56.299789 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260812 03:12:56.299991 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.290274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51180 (local address 127.26.250.189:41353)
0812 03:12:56.290434 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:56.290439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:56.290457 (+ 18us) server_negotiation.cc:407] Connection header received
0812 03:12:56.290546 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:56.290550 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:56.290605 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:56.290699 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:56.291198 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.292028 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.292810 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.293240 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.293376 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:56.296162 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:56.296167 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:56.296170 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:56.296199 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:56.297975 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.298622 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.298627 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.298630 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.298755 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.299074 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.299077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.299079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.299291 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:56.299402 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:56.299683 (+ 281us) server_negotiation.cc:299] Negotiation successful
0812 03:12:56.299850 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":51}
I20260812 03:12:56.301339 19386 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:12:56.302332 19390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.301938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:42319 (local address 127.0.0.1:54904)
0812 03:12:56.302094 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.302213 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:42319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260812 03:12:56.302685 19331 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.26.250.188:42319: connect: Connection refused (error 111)
I20260812 03:12:56.303673 19390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.303435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:42319 (local address 127.0.0.1:54918)
0812 03:12:56.303552 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.303602 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:42319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260812 03:12:56.303894 19386 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.188:42319: Network error: Client connection negotiation failed: client connection to 127.26.250.188:42319: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.26.250.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.188.keytab.
Entry for principal kudu/127.26.250.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.188.keytab.
I20260812 03:12:56.310298 27626 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
/tmp/dist-test-taskh6ikwC/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.188:42319
--webserver_interface=127.26.250.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.188.keytab
--principal=kudu/127.26.250.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.26.250.148:42527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.26.250.190:43041,127.26.250.189:41353,127.26.250.188:42319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 03:12:56.323122 19304 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } attempt: 1
I20260812 03:12:56.324054 19392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.323716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:42319 (local address 127.0.0.1:54920)
0812 03:12:56.323899 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.323972 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:42319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260812 03:12:56.324785 19392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.324566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:42319 (local address 127.0.0.1:54924)
0812 03:12:56.324674 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.324724 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:42319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 03:12:56.324921 19304 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.188:42319: Network error: Client connection negotiation failed: client connection to 127.26.250.188:42319: connect: Connection refused (error 111)
I20260812 03:12:56.353196 19386 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } attempt: 1
I20260812 03:12:56.354233 19390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.353901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:42319 (local address 127.0.0.1:54940)
0812 03:12:56.354083 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.354152 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:42319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260812 03:12:56.355048 19390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.354810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:42319 (local address 127.0.0.1:54954)
0812 03:12:56.354935 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.354992 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:42319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 03:12:56.355178 19386 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.188:42319: Network error: Client connection negotiation failed: client connection to 127.26.250.188:42319: connect: Connection refused (error 111)
I20260812 03:12:56.400112 19304 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } attempt: 2
I20260812 03:12:56.401221 19392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.400874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:42319 (local address 127.0.0.1:54970)
0812 03:12:56.401058 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.401132 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:42319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260812 03:12:56.402076 19392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.401856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:42319 (local address 127.0.0.1:54980)
0812 03:12:56.401963 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.402023 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:42319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 03:12:56.402215 19304 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.188:42319: Network error: Client connection negotiation failed: client connection to 127.26.250.188:42319: connect: Connection refused (error 111)
I20260812 03:12:56.423368 19386 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } attempt: 2
I20260812 03:12:56.424422 19390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.423997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:42319 (local address 127.0.0.1:54988)
0812 03:12:56.424268 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.424339 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:42319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 03:12:56.425331 19390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.424989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:42319 (local address 127.0.0.1:55002)
0812 03:12:56.425117 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.425267 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:42319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 03:12:56.425473 19386 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.188:42319: Network error: Client connection negotiation failed: client connection to 127.26.250.188:42319: connect: Connection refused (error 111)
W20260812 03:12:56.436592 19394 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 03:12:56.437008 19394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 03:12:56.437080 19394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 03:12:56.437162 19394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 03:12:56.442070 19394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 03:12:56.442198 19394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 03:12:56.442245 19394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 03:12:56.442282 19394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 03:12:56.442314 19394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 03:12:56.448204 19394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.26.250.148:42527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.26.250.190:43041,127.26.250.189:41353,127.26.250.188:42319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/krb5kdc/kudu_127.26.250.188.keytab
--principal=kudu/127.26.250.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.26.250.188:42319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.26.250.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 12 Aug 2026 02:22:35 UTC on ccycloud.quasar-gitnpj.root.comops.site
build id 4581827
I20260812 03:12:56.449553 19394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 03:12:56.450732 19394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 03:12:56.460153 19402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:56.460109 19404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 03:12:56.460109 19401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 03:12:56.460435 19394 server_base.cc:1034] running on GCE node
I20260812 03:12:56.461153 19394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 03:12:56.462502 19394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 03:12:56.463717 19394 hybrid_clock.cc:648] HybridClock initialized: now 1786504376463692 us; error 37 us; skew 500 ppm
Aug 12 03:12:56 dist-test-slave-zpfg krb5kdc[19213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504376, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 03:12:56.466954 19394 init.cc:377] Logged in from keytab as kudu/127.26.250.188@KRBTEST.COM (short username kudu)
I20260812 03:12:56.468061 19394 webserver.cc:460] Webserver started at http://127.26.250.188:36555/ using document root <none> and password file <none>
I20260812 03:12:56.468693 19394 fs_manager.cc:359] Metadata directory not provided
I20260812 03:12:56.468796 19394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 03:12:56.469048 19394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 03:12:56.471347 19394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-2/data/instance:
uuid: "f66d123298b6409887bef8e582c56a39"
format_stamp: "Formatted at 2026-08-12 03:12:56 on dist-test-slave-zpfg"
server_key: "2164da51a971550005d78e4a596e4c94"
server_key_iv: "97e70c26cb89e6fd92e646919e5dfff0"
server_key_version: "encryptionkey@0"
I20260812 03:12:56.471967 19394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-2/wal/instance:
uuid: "f66d123298b6409887bef8e582c56a39"
format_stamp: "Formatted at 2026-08-12 03:12:56 on dist-test-slave-zpfg"
server_key: "2164da51a971550005d78e4a596e4c94"
server_key_iv: "97e70c26cb89e6fd92e646919e5dfff0"
server_key_version: "encryptionkey@0"
I20260812 03:12:56.476280 19394 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 03:12:56.479351 19411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 03:12:56.480755 19394 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 03:12:56.480949 19394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-2/data,/tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-2/wal
uuid: "f66d123298b6409887bef8e582c56a39"
format_stamp: "Formatted at 2026-08-12 03:12:56 on dist-test-slave-zpfg"
server_key: "2164da51a971550005d78e4a596e4c94"
server_key_iv: "97e70c26cb89e6fd92e646919e5dfff0"
server_key_version: "encryptionkey@0"
I20260812 03:12:56.481083 19394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 03:12:56.501403 19304 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } attempt: 3
I20260812 03:12:56.502295 19392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.501974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:42319 (local address 127.0.0.1:55006)
0812 03:12:56.502139 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.502205 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:42319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260812 03:12:56.503087 19392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.502889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:42319 (local address 127.0.0.1:55010)
0812 03:12:56.502987 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.503035 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.26.250.188:42319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 03:12:56.503226 19304 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.26.250.188:42319: Network error: Client connection negotiation failed: client connection to 127.26.250.188:42319: connect: Connection refused (error 111)
I20260812 03:12:56.503311 19394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 03:12:56.507139 19394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 03:12:56.507360 19394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 03:12:56.515321 19394 rpc_server.cc:307] RPC server started. Bound to: 127.26.250.188:42319
I20260812 03:12:56.515655 19466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.26.250.188:42319 every 8 connection(s)
I20260812 03:12:56.516386 19394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-2/data/info.pb
I20260812 03:12:56.516659 19386 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } attempt: 3
I20260812 03:12:56.519209 27626 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu as pid 19394
I20260812 03:12:56.519384 27626 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskh6ikwC/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786504075277362-27626-0/minicluster-data/master-2/wal/instance
I20260812 03:12:56.519747 19467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 03:12:56.519764 27626 external_mini_cluster.cc:1428] Setting key 0b4ef07b835b7f2a2ffda460734466be
I20260812 03:12:56.524448 19467 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43041 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 03:12:56 dist-test-slave-zpfg krb5kdc[19213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504376, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.189@KRBTEST.COM for kudu/127.26.250.188@KRBTEST.COM
Aug 12 03:12:56 dist-test-slave-zpfg krb5kdc[19213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.188@KRBTEST.COM
I20260812 03:12:56.531159 19390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.517200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:42319 (local address 127.0.0.1:55018)
0812 03:12:56.517424 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.517445 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:56.517567 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:56.520440 (+ 2873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:56.520443 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:56.520647 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:56.520797 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:56.520806 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.522156 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.522163 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:56.523020 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:56.523040 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.524024 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.524028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:56.524119 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:56.524742 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:56.524758 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:56.526333 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:56.528972 (+ 2639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:56.528976 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:56.528978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:56.529267 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:56.529724 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:56.529727 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:56.529730 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:56.529838 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:56.530638 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:56.530641 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:56.530750 (+ 109us) client_negotiation.cc:769] Sending connection context
0812 03:12:56.530821 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 03:12:56.530978 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260812 03:12:56.533208 19469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.517995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55018 (local address 127.26.250.188:42319)
0812 03:12:56.518533 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:56.518550 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:56.518588 (+ 38us) server_negotiation.cc:407] Connection header received
0812 03:12:56.519610 (+ 1022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:56.519637 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:56.520109 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:56.520495 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:56.520939 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.522010 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.523165 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.523885 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.524057 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:56.526596 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:56.526601 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:56.526615 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:56.526640 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:56.528801 (+ 2161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.529416 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.529422 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.529428 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.529598 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.529940 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.529946 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.529948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.530495 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:56.531452 (+ 957us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:56.531651 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 03:12:56.531976 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":147,"threads_started":1}
Aug 12 03:12:56 dist-test-slave-zpfg krb5kdc[19213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504376, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.188@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:12:56.533769 19471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.520738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55028 (local address 127.26.250.188:42319)
0812 03:12:56.521055 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:56.521066 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:56.522500 (+ 1434us) server_negotiation.cc:407] Connection header received
0812 03:12:56.522629 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:56.522632 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:56.522679 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:56.522778 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:56.523670 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.524364 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.525575 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.525810 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.525919 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:56.529450 (+ 3531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:56.529453 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:56.529455 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:56.529480 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:56.531498 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.531949 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.531953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.531955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.532072 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.532433 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.532436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.532437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.532606 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:56.532799 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:56.533052 (+ 253us) server_negotiation.cc:299] Negotiation successful
0812 03:12:56.533201 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":87,"threads_started":1}
I20260812 03:12:56.538285 19308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.526788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36600 (local address 127.26.250.190:43041)
0812 03:12:56.526937 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:56.526945 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:56.527402 (+ 457us) server_negotiation.cc:407] Connection header received
0812 03:12:56.527576 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:56.527581 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:56.527637 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:56.527706 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:56.528575 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.529248 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.530437 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.530843 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.530984 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:56.534191 (+ 3207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:56.534195 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:56.534197 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:56.534225 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:56.535856 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.536440 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.536443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.536445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.536554 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.536944 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.536947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.536949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.537270 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:56.537349 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:56.537965 (+ 616us) server_negotiation.cc:299] Negotiation successful
0812 03:12:56.538095 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 03:12:56.538915 19472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.526619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.190:43041 (local address 127.0.0.1:36600)
0812 03:12:56.527272 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.527330 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:56.527456 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:56.527860 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:56.527866 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:56.528265 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:56.528423 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:56.528431 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.529366 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.529371 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:56.530288 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:56.530303 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.531007 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.531011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:56.531121 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:56.531651 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:56.531665 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:56.534035 (+ 2370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:56.535988 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:56.536003 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:56.536010 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:56.536341 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:56.536687 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:56.536691 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:56.536693 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:56.536815 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:56.537493 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:56.537500 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:56.537636 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 03:12:56.538443 (+ 807us) client_negotiation.cc:240] Negotiation successful
0812 03:12:56.538734 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":138,"threads_started":1}
I20260812 03:12:56.539381 19386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 420e374d01ec44f9a527b444de342b8b: Bootstrap starting.
I20260812 03:12:56.540439 19467 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.189" port: 41353 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:12:56.542668 19386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 420e374d01ec44f9a527b444de342b8b: Neither blocks nor log segments found. Creating new log.
Aug 12 03:12:56 dist-test-slave-zpfg krb5kdc[19213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.190@KRBTEST.COM
I20260812 03:12:56.543608 19386 log.cc:824] T 00000000000000000000000000000000 P 420e374d01ec44f9a527b444de342b8b: Log is configured to *not* fsync() on all Append() calls
Aug 12 03:12:56 dist-test-slave-zpfg krb5kdc[19213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504376, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.188@KRBTEST.COM for kudu/127.26.250.189@KRBTEST.COM
I20260812 03:12:56.546870 19386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 420e374d01ec44f9a527b444de342b8b: No bootstrap required, opened a new log
I20260812 03:12:56.548146 19473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.537175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36608 (local address 127.26.250.190:43041)
0812 03:12:56.537452 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:56.537465 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:56.537477 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:12:56.537513 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:56.537521 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:56.537556 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:56.537664 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:56.538551 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.539354 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.540431 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.540766 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.540916 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:56.543931 (+ 3015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:56.543935 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:56.543937 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:56.543962 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:56.545662 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.546109 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.546113 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.546114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.546205 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.546645 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.546648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.546650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.546892 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:56.547137 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:56.547791 (+ 654us) server_negotiation.cc:299] Negotiation successful
0812 03:12:56.547935 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":84,"threads_started":1}
I20260812 03:12:56.551354 19472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.541026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.189:41353 (local address 127.0.0.1:51194)
0812 03:12:56.541211 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.541443 (+ 232us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:56.541697 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:56.542000 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:56.542006 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:56.542205 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:56.542402 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:56.542413 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.543413 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.543416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:56.544213 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:56.544232 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.545019 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.545023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:56.545142 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:56.545607 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:56.545622 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:56.547173 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:56.549452 (+ 2279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:56.549455 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:56.549457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:56.549746 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:56.550103 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:56.550106 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:56.550109 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:56.550204 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:56.550775 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:56.550779 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:56.550905 (+ 126us) client_negotiation.cc:769] Sending connection context
0812 03:12:56.551019 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 03:12:56.551187 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260812 03:12:56.551375 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.541435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51194 (local address 127.26.250.189:41353)
0812 03:12:56.541585 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:56.541590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:56.541606 (+ 16us) server_negotiation.cc:407] Connection header received
0812 03:12:56.541817 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:56.541821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:56.541869 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:56.541955 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:56.542539 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.543272 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.544394 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.544744 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.544850 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:56.547312 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:56.547315 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:56.547318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:56.547354 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:56.549314 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.549870 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.549877 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.549881 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.549976 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.550316 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.550320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.550322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.550651 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:56.550759 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:56.551104 (+ 345us) server_negotiation.cc:299] Negotiation successful
0812 03:12:56.551226 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":62}
I20260812 03:12:56.551836 19386 raft_consensus.cc:348] T 00000000000000000000000000000000 P 420e374d01ec44f9a527b444de342b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d033b584459d464f8265d1e3f6fb378f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43041 } } peers { permanent_uuid: "420e374d01ec44f9a527b444de342b8b" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 41353 } } peers { permanent_uuid: "f66d123298b6409887bef8e582c56a39" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } }
I20260812 03:12:56.552078 19386 raft_consensus.cc:729] T 00000000000000000000000000000000 P 420e374d01ec44f9a527b444de342b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 420e374d01ec44f9a527b444de342b8b, State: Initialized, Role: FOLLOWER
I20260812 03:12:56.552464 19386 consensus_queue.cc:260] T 00000000000000000000000000000000 P 420e374d01ec44f9a527b444de342b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d033b584459d464f8265d1e3f6fb378f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43041 } } peers { permanent_uuid: "420e374d01ec44f9a527b444de342b8b" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 41353 } } peers { permanent_uuid: "f66d123298b6409887bef8e582c56a39" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } }
I20260812 03:12:56.552600 19467 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } has no permanent_uuid. Determining permanent_uuid...
I20260812 03:12:56.552977 19476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 420e374d01ec44f9a527b444de342b8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d033b584459d464f8265d1e3f6fb378f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43041 } } peers { permanent_uuid: "420e374d01ec44f9a527b444de342b8b" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 41353 } } peers { permanent_uuid: "f66d123298b6409887bef8e582c56a39" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } } }
I20260812 03:12:56.553817 19476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 420e374d01ec44f9a527b444de342b8b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:12:56.554325 19386 sys_catalog.cc:564] T 00000000000000000000000000000000 P 420e374d01ec44f9a527b444de342b8b [sys.catalog]: configured and running, proceeding with master startup.
Aug 12 03:12:56 dist-test-slave-zpfg krb5kdc[19213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.26.250.189@KRBTEST.COM
Aug 12 03:12:56 dist-test-slave-zpfg krb5kdc[19213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504376, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.188@KRBTEST.COM for kudu/127.26.250.188@KRBTEST.COM
W20260812 03:12:56.560940 19489 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 420e374d01ec44f9a527b444de342b8b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:12:56.561076 19489 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:12:56.561967 19475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.549707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51198 (local address 127.26.250.189:41353)
0812 03:12:56.550161 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:56.550170 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:56.550182 (+ 12us) server_negotiation.cc:407] Connection header received
0812 03:12:56.550214 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:56.550216 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:56.550270 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:56.550388 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:56.551146 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.552051 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.553081 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.553428 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.553537 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:56.556982 (+ 3445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:56.556986 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:56.556988 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:56.557014 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:56.559504 (+ 2490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.560013 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.560016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.560018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.560106 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.560444 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.560447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.560449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.560689 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:56.560898 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:56.561642 (+ 744us) server_negotiation.cc:299] Negotiation successful
0812 03:12:56.561781 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":380,"thread_start_us":173,"threads_started":1}
I20260812 03:12:56.562312 19469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.553351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55042 (local address 127.26.250.188:42319)
0812 03:12:56.553442 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:56.553446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:56.553459 (+ 13us) server_negotiation.cc:407] Connection header received
0812 03:12:56.553603 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:56.553607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:56.553647 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:56.553694 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:56.554205 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.554837 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.555774 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.556010 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.556126 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:56.558748 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:56.558753 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:56.558755 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:56.558789 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:56.560483 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.561065 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.561068 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.561070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.561140 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.561386 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.561389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.561391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.561672 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:56.561743 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:56.562045 (+ 302us) server_negotiation.cc:299] Negotiation successful
0812 03:12:56.562158 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 03:12:56.562811 19472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.553096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:42319 (local address 127.0.0.1:55042)
0812 03:12:56.553316 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.553329 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:56.553514 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:56.553835 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:56.553840 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:56.554018 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:56.554090 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:56.554095 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.554951 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.554954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:56.555659 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:56.555670 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.556269 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.556276 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:56.556369 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:56.556848 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:56.556863 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:56.558608 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:56.560637 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:56.560640 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:56.560642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:56.560964 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:56.561232 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:56.561234 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:56.561236 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:56.561305 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:56.561814 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:56.561816 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:56.561897 (+ 81us) client_negotiation.cc:769] Sending connection context
0812 03:12:56.562521 (+ 624us) client_negotiation.cc:240] Negotiation successful
0812 03:12:56.562652 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158}
I20260812 03:12:56.560158 19312 master_runner.cc:186] Error getting master registration for 127.26.250.190:43041: OK, instance_id { permanent_uuid: "d033b584459d464f8265d1e3f6fb378f" instance_seqno: 1786504376022190 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:12:56.563871 19312 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:12:56.565645 19467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f66d123298b6409887bef8e582c56a39: Bootstrap starting.
I20260812 03:12:56.568372 19467 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f66d123298b6409887bef8e582c56a39: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:56.568612 27626 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 03:12:56.569332 19467 log.cc:824] T 00000000000000000000000000000000 P f66d123298b6409887bef8e582c56a39: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:56.570005 19275 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d033b584459d464f8265d1e3f6fb378f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36608
I20260812 03:12:56.572072 19467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f66d123298b6409887bef8e582c56a39: No bootstrap required, opened a new log
I20260812 03:12:56.576489 19467 raft_consensus.cc:348] T 00000000000000000000000000000000 P f66d123298b6409887bef8e582c56a39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d033b584459d464f8265d1e3f6fb378f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43041 } } peers { permanent_uuid: "420e374d01ec44f9a527b444de342b8b" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 41353 } } peers { permanent_uuid: "f66d123298b6409887bef8e582c56a39" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } }
I20260812 03:12:56.576815 19467 raft_consensus.cc:729] T 00000000000000000000000000000000 P f66d123298b6409887bef8e582c56a39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f66d123298b6409887bef8e582c56a39, State: Initialized, Role: FOLLOWER
I20260812 03:12:56.577219 19467 consensus_queue.cc:260] T 00000000000000000000000000000000 P f66d123298b6409887bef8e582c56a39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d033b584459d464f8265d1e3f6fb378f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43041 } } peers { permanent_uuid: "420e374d01ec44f9a527b444de342b8b" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 41353 } } peers { permanent_uuid: "f66d123298b6409887bef8e582c56a39" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } }
I20260812 03:12:56.577766 19492 sys_catalog.cc:455] T 00000000000000000000000000000000 P f66d123298b6409887bef8e582c56a39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d033b584459d464f8265d1e3f6fb378f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43041 } } peers { permanent_uuid: "420e374d01ec44f9a527b444de342b8b" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 41353 } } peers { permanent_uuid: "f66d123298b6409887bef8e582c56a39" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } } }
I20260812 03:12:56.577912 19492 sys_catalog.cc:458] T 00000000000000000000000000000000 P f66d123298b6409887bef8e582c56a39 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:12:56.578718 19467 sys_catalog.cc:564] T 00000000000000000000000000000000 P f66d123298b6409887bef8e582c56a39 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 03:12:56.584672 19508 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f66d123298b6409887bef8e582c56a39: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:12:56.584754 19508 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:12:56.583226 19394 master_runner.cc:186] Error getting master registration for 127.26.250.190:43041: OK, instance_id { permanent_uuid: "d033b584459d464f8265d1e3f6fb378f" instance_seqno: 1786504376022190 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 03:12:56.586397 19394 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:12:56.667382 19304 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } attempt: 4
Aug 12 03:12:56 dist-test-slave-zpfg krb5kdc[19213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786504375, etypes {rep=17 tkt=17 ses=17}, kudu/127.26.250.190@KRBTEST.COM for kudu/127.26.250.188@KRBTEST.COM
I20260812 03:12:56.677449 19469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.668190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55048 (local address 127.26.250.188:42319)
0812 03:12:56.668322 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:56.668328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:56.668552 (+ 224us) server_negotiation.cc:407] Connection header received
0812 03:12:56.668606 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:56.668608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:56.668661 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:56.668736 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:56.669289 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.670094 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.670987 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.671372 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.671469 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:56.673905 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:56.673909 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:56.673911 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:56.673936 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:56.675707 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.676163 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.676168 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.676170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.676259 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.676562 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.676567 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.676569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.676797 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:56.676898 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:56.677167 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 03:12:56.677287 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:12:56.677464 19392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.668100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.26.250.188:42319 (local address 127.0.0.1:55048)
0812 03:12:56.668321 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0812 03:12:56.668336 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 03:12:56.668453 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 03:12:56.668766 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 03:12:56.668768 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 03:12:56.669023 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0812 03:12:56.669151 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:56.669157 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.670203 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.670207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:56.670861 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 03:12:56.670870 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.671498 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.671502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 03:12:56.671597 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:56.672183 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 03:12:56.672195 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0812 03:12:56.673751 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 03:12:56.675807 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:56.675811 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:56.675813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:56.676031 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:56.676365 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 03:12:56.676369 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 03:12:56.676371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 03:12:56.676464 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 03:12:56.676898 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 03:12:56.676901 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 03:12:56.676999 (+ 98us) client_negotiation.cc:769] Sending connection context
0812 03:12:56.677111 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 03:12:56.677272 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139}
I20260812 03:12:56.681536 19304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f: Bootstrap starting.
I20260812 03:12:56.684060 19304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f: Neither blocks nor log segments found. Creating new log.
I20260812 03:12:56.685177 19304 log.cc:824] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f: Log is configured to *not* fsync() on all Append() calls
I20260812 03:12:56.687469 19304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f: No bootstrap required, opened a new log
I20260812 03:12:56.690207 19304 raft_consensus.cc:348] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d033b584459d464f8265d1e3f6fb378f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43041 } } peers { permanent_uuid: "420e374d01ec44f9a527b444de342b8b" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 41353 } } peers { permanent_uuid: "f66d123298b6409887bef8e582c56a39" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } }
I20260812 03:12:56.690510 19304 raft_consensus.cc:729] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d033b584459d464f8265d1e3f6fb378f, State: Initialized, Role: FOLLOWER
I20260812 03:12:56.690891 19304 consensus_queue.cc:260] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d033b584459d464f8265d1e3f6fb378f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43041 } } peers { permanent_uuid: "420e374d01ec44f9a527b444de342b8b" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 41353 } } peers { permanent_uuid: "f66d123298b6409887bef8e582c56a39" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } }
I20260812 03:12:56.691495 19511 sys_catalog.cc:455] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d033b584459d464f8265d1e3f6fb378f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43041 } } peers { permanent_uuid: "420e374d01ec44f9a527b444de342b8b" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 41353 } } peers { permanent_uuid: "f66d123298b6409887bef8e582c56a39" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } } }
I20260812 03:12:56.691766 19511 sys_catalog.cc:458] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:12:56.692394 19304 sys_catalog.cc:564] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 03:12:56.692461 19275 raft_consensus.cc:482] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 03:12:56.692554 19275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:56.694007 19275 raft_consensus.cc:504] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d033b584459d464f8265d1e3f6fb378f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43041 } } peers { permanent_uuid: "420e374d01ec44f9a527b444de342b8b" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 41353 } } peers { permanent_uuid: "f66d123298b6409887bef8e582c56a39" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } }
I20260812 03:12:56.695108 19275 leader_election.cc:288] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f [CANDIDATE]: Term 1 election: Requested vote from peers 420e374d01ec44f9a527b444de342b8b (127.26.250.189:41353), f66d123298b6409887bef8e582c56a39 (127.26.250.188:42319)
I20260812 03:12:56.697814 19359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d033b584459d464f8265d1e3f6fb378f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "420e374d01ec44f9a527b444de342b8b"
I20260812 03:12:56.698073 19359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 420e374d01ec44f9a527b444de342b8b [term 0 FOLLOWER]: Advancing to term 1
W20260812 03:12:56.698232 19522 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 03:12:56.698297 19522 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 03:12:56.700250 19440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d033b584459d464f8265d1e3f6fb378f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f66d123298b6409887bef8e582c56a39"
I20260812 03:12:56.700543 19440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f66d123298b6409887bef8e582c56a39 [term 0 FOLLOWER]: Advancing to term 1
I20260812 03:12:56.702070 19440 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f66d123298b6409887bef8e582c56a39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d033b584459d464f8265d1e3f6fb378f in term 1.
I20260812 03:12:56.702714 19247 leader_election.cc:302] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d033b584459d464f8265d1e3f6fb378f, f66d123298b6409887bef8e582c56a39; no voters:
I20260812 03:12:56.705850 19359 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 420e374d01ec44f9a527b444de342b8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d033b584459d464f8265d1e3f6fb378f in term 1.
I20260812 03:12:56.710075 19511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 03:12:56.710484 19511 raft_consensus.cc:686] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f [term 1 LEADER]: Becoming Leader. State: Replica: d033b584459d464f8265d1e3f6fb378f, State: Running, Role: LEADER
I20260812 03:12:56.710919 19511 consensus_queue.cc:237] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d033b584459d464f8265d1e3f6fb378f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43041 } } peers { permanent_uuid: "420e374d01ec44f9a527b444de342b8b" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 41353 } } peers { permanent_uuid: "f66d123298b6409887bef8e582c56a39" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } }
I20260812 03:12:56.712150 19227 master_runner.cc:263] No leader master found from master d033b584459d464f8265d1e3f6fb378f
I20260812 03:12:56.712234 19227 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 03:12:56.713186 19511 sys_catalog.cc:455] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d033b584459d464f8265d1e3f6fb378f. Latest consensus state: current_term: 1 leader_uuid: "d033b584459d464f8265d1e3f6fb378f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d033b584459d464f8265d1e3f6fb378f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43041 } } peers { permanent_uuid: "420e374d01ec44f9a527b444de342b8b" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 41353 } } peers { permanent_uuid: "f66d123298b6409887bef8e582c56a39" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } } }
I20260812 03:12:56.713419 19511 sys_catalog.cc:458] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:56.715860 19530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 03:12:56.716912 19530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 03:12:56.731206 19440 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f66d123298b6409887bef8e582c56a39 [term 1 FOLLOWER]: Refusing update from remote peer d033b584459d464f8265d1e3f6fb378f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:12:56.732357 19528 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f66d123298b6409887bef8e582c56a39" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:12:56.737155 19492 sys_catalog.cc:455] T 00000000000000000000000000000000 P f66d123298b6409887bef8e582c56a39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d033b584459d464f8265d1e3f6fb378f. Latest consensus state: current_term: 1 leader_uuid: "d033b584459d464f8265d1e3f6fb378f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d033b584459d464f8265d1e3f6fb378f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43041 } } peers { permanent_uuid: "420e374d01ec44f9a527b444de342b8b" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 41353 } } peers { permanent_uuid: "f66d123298b6409887bef8e582c56a39" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } } }
I20260812 03:12:56.737314 19492 sys_catalog.cc:458] T 00000000000000000000000000000000 P f66d123298b6409887bef8e582c56a39 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:12:56.738868 19359 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 420e374d01ec44f9a527b444de342b8b [term 1 FOLLOWER]: Refusing update from remote peer d033b584459d464f8265d1e3f6fb378f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 03:12:56.739662 19528 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f [LEADER]: Connected to new peer: Peer: permanent_uuid: "420e374d01ec44f9a527b444de342b8b" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 41353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 03:12:56.744504 19476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 420e374d01ec44f9a527b444de342b8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d033b584459d464f8265d1e3f6fb378f. Latest consensus state: current_term: 1 leader_uuid: "d033b584459d464f8265d1e3f6fb378f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d033b584459d464f8265d1e3f6fb378f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43041 } } peers { permanent_uuid: "420e374d01ec44f9a527b444de342b8b" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 41353 } } peers { permanent_uuid: "f66d123298b6409887bef8e582c56a39" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } } }
I20260812 03:12:56.744668 19476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 420e374d01ec44f9a527b444de342b8b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:12:56.748961 19469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.715245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55056 (local address 127.26.250.188:42319)
0812 03:12:56.715376 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:56.715381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:56.725923 (+ 10542us) server_negotiation.cc:407] Connection header received
0812 03:12:56.726097 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:56.726102 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:56.726158 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:56.726212 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:56.728442 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.729188 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.733489 (+ 4301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.733794 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.733888 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:56.741911 (+ 8023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:56.741916 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:56.741919 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:56.741952 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:56.743766 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.744989 (+ 1223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.744994 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.744996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.745095 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.746465 (+ 1370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.746470 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.746472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.746707 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:56.746856 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:56.748641 (+ 1785us) server_negotiation.cc:299] Negotiation successful
0812 03:12:56.748787 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 03:12:56.749825 19473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.714092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36614 (local address 127.26.250.190:43041)
0812 03:12:56.714233 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:56.714238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:56.720833 (+ 6595us) server_negotiation.cc:407] Connection header received
0812 03:12:56.721047 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:56.721050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:56.721101 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:56.721154 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:56.722239 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.723122 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.724395 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.724780 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.724860 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:56.736812 (+ 11952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:56.736819 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:56.736822 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:56.736886 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:56.739252 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.745454 (+ 6202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.745459 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.745461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.745578 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.747204 (+ 1626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.747208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.747210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.747454 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:56.747574 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:56.749482 (+ 1908us) server_negotiation.cc:299] Negotiation successful
0812 03:12:56.749632 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 03:12:56.749938 19528 sys_catalog.cc:455] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d033b584459d464f8265d1e3f6fb378f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d033b584459d464f8265d1e3f6fb378f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43041 } } peers { permanent_uuid: "420e374d01ec44f9a527b444de342b8b" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 41353 } } peers { permanent_uuid: "f66d123298b6409887bef8e582c56a39" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } } }
I20260812 03:12:56.750059 19528 sys_catalog.cc:458] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:56.750713 19475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:56.714667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51204 (local address 127.26.250.189:41353)
0812 03:12:56.714789 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:56.714793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:56.725645 (+ 10852us) server_negotiation.cc:407] Connection header received
0812 03:12:56.725862 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:56.725869 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:56.725934 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:56.725989 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:56.730421 (+ 4432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.731233 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.732292 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:56.732646 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:56.732738 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:56.739461 (+ 6723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 03:12:56.739464 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 03:12:56.739467 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 03:12:56.739498 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 03:12:56.741261 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.745725 (+ 4464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.745730 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.745731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.745844 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 03:12:56.747907 (+ 2063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 03:12:56.747911 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 03:12:56.747913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 03:12:56.748161 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 03:12:56.748287 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:56.750386 (+ 2099us) server_negotiation.cc:299] Negotiation successful
0812 03:12:56.750556 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 03:12:56.754266 19492 sys_catalog.cc:455] T 00000000000000000000000000000000 P f66d123298b6409887bef8e582c56a39 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d033b584459d464f8265d1e3f6fb378f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d033b584459d464f8265d1e3f6fb378f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43041 } } peers { permanent_uuid: "420e374d01ec44f9a527b444de342b8b" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 41353 } } peers { permanent_uuid: "f66d123298b6409887bef8e582c56a39" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } } }
I20260812 03:12:56.754442 19492 sys_catalog.cc:458] T 00000000000000000000000000000000 P f66d123298b6409887bef8e582c56a39 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 03:12:56.756323 19335 tablet.cc:2369] T 00000000000000000000000000000000 P 420e374d01ec44f9a527b444de342b8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 03:12:56.756917 19534 mvcc.cc:205] Tried to move back new op lower bound from 7317521927080546304 to 7317521927012560896. Current Snapshot: MvccSnapshot[applied={T|T < 7317521927080546304}]
I20260812 03:12:56.757495 19535 mvcc.cc:205] Tried to move back new op lower bound from 7317521927080546304 to 7317521927012560896. Current Snapshot: MvccSnapshot[applied={T|T < 7317521927080546304}]
I20260812 03:12:56.767656 19511 sys_catalog.cc:455] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d033b584459d464f8265d1e3f6fb378f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d033b584459d464f8265d1e3f6fb378f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43041 } } peers { permanent_uuid: "420e374d01ec44f9a527b444de342b8b" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 41353 } } peers { permanent_uuid: "f66d123298b6409887bef8e582c56a39" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } } }
I20260812 03:12:56.767819 19511 sys_catalog.cc:458] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f [sys.catalog]: This master's current role is: LEADER
I20260812 03:12:56.772254 19476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 420e374d01ec44f9a527b444de342b8b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d033b584459d464f8265d1e3f6fb378f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d033b584459d464f8265d1e3f6fb378f" member_type: VOTER last_known_addr { host: "127.26.250.190" port: 43041 } } peers { permanent_uuid: "420e374d01ec44f9a527b444de342b8b" member_type: VOTER last_known_addr { host: "127.26.250.189" port: 41353 } } peers { permanent_uuid: "f66d123298b6409887bef8e582c56a39" member_type: VOTER last_known_addr { host: "127.26.250.188" port: 42319 } } }
I20260812 03:12:56.772414 19476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 420e374d01ec44f9a527b444de342b8b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 03:12:56.782605 19530 catalog_manager.cc:1347] Generated new cluster ID: a962f6387f45481b98c16b7fcb28233d
I20260812 03:12:56.782709 19530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 03:12:56.785094 19537 mvcc.cc:205] Tried to move back new op lower bound from 7317521927080546304 to 7317521927012560896. Current Snapshot: MvccSnapshot[applied={T|T < 7317521927080546304 or (T in {7317521927080546304})}]
I20260812 03:12:56.809966 19530 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 03:12:56.811317 19530 catalog_manager.cc:1504] Loading token signing keys...
I20260812 03:12:56.827085 19530 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d033b584459d464f8265d1e3f6fb378f: Generated new TSK 0
I20260812 03:12:56.828065 19530 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 03:12:56.855058 19473 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:12:56.853130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36616 (local address 127.26.250.190:43041)
0812 03:12:56.853284 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:56.853289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:56.853546 (+ 257us) server_negotiation.cc:407] Connection header received
0812 03:12:56.853969 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:56.853976 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:56.854041 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:56.854096 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:56.854916 (+ 820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36616: BlockingRecv error: recv got EOF from 127.0.0.1:36616 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260812 03:12:56.855058 19469 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:12:56.853676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55072 (local address 127.26.250.188:42319)
0812 03:12:56.853819 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:56.853826 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:56.853886 (+ 60us) server_negotiation.cc:407] Connection header received
0812 03:12:56.854049 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:56.854052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:56.854103 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:56.854153 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:56.854916 (+ 763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55072: BlockingRecv error: recv got EOF from 127.0.0.1:55072 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260812 03:12:56.855090 19475 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:12:56.853184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51220 (local address 127.26.250.189:41353)
0812 03:12:56.853338 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:56.853344 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:56.853872 (+ 528us) server_negotiation.cc:407] Connection header received
0812 03:12:56.854089 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:56.854092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:56.854145 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:56.854197 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:56.854940 (+ 743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51220: BlockingRecv error: recv got EOF from 127.0.0.1:51220 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260812 03:12:56.858500 19475 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:12:56.857336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51232 (local address 127.26.250.189:41353)
0812 03:12:56.857448 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:56.857452 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:56.857586 (+ 134us) server_negotiation.cc:407] Connection header received
0812 03:12:56.857786 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:56.857788 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:56.857853 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:56.857945 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:56.858387 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51232: BlockingRecv error: recv got EOF from 127.0.0.1:51232 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260812 03:12:56.862596 19475 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:12:56.861644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51234 (local address 127.26.250.189:41353)
0812 03:12:56.861789 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:56.861793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:56.861807 (+ 14us) server_negotiation.cc:407] Connection header received
0812 03:12:56.861853 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:56.861856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:56.861907 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:56.862001 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:56.862487 (+ 486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51234: BlockingRecv error: recv got EOF from 127.0.0.1:51234 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260812 03:12:56.867465 19475 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:12:56.866433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51248 (local address 127.26.250.189:41353)
0812 03:12:56.866556 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:56.866561 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:56.866696 (+ 135us) server_negotiation.cc:407] Connection header received
0812 03:12:56.866789 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:56.866792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:56.866847 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:56.866939 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:56.867351 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51248: BlockingRecv error: recv got EOF from 127.0.0.1:51248 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260812 03:12:56.874617 19475 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:12:56.873434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51252 (local address 127.26.250.189:41353)
0812 03:12:56.873624 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:56.873628 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:56.873696 (+ 68us) server_negotiation.cc:407] Connection header received
0812 03:12:56.873846 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:56.873850 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:56.873917 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:56.874020 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:56.874494 (+ 474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51252: BlockingRecv error: recv got EOF from 127.0.0.1:51252 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
W20260812 03:12:56.886628 19475 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:12:56.885014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51264 (local address 127.26.250.189:41353)
0812 03:12:56.885334 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:56.885338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:56.885604 (+ 266us) server_negotiation.cc:407] Connection header received
0812 03:12:56.885781 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:56.885784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:56.885845 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:56.885948 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:56.886492 (+ 544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51264: BlockingRecv error: recv got EOF from 127.0.0.1:51264 (error 108)
Metrics: {"server-negotiator.queue_time_us":215}
W20260812 03:12:56.906121 19475 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:12:56.904910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51272 (local address 127.26.250.189:41353)
0812 03:12:56.905068 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:56.905073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:56.905183 (+ 110us) server_negotiation.cc:407] Connection header received
0812 03:12:56.905329 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:56.905342 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:56.905402 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:56.905493 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:56.905991 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51272: BlockingRecv error: recv got EOF from 127.0.0.1:51272 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260812 03:12:56.941813 19475 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:12:56.940666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51274 (local address 127.26.250.189:41353)
0812 03:12:56.940863 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:56.940866 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:56.940908 (+ 42us) server_negotiation.cc:407] Connection header received
0812 03:12:56.941058 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:56.941061 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:56.941113 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:56.941188 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:56.941659 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51274: BlockingRecv error: recv got EOF from 127.0.0.1:51274 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260812 03:12:57.009455 19475 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:12:57.008048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51288 (local address 127.26.250.189:41353)
0812 03:12:57.008329 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:57.008334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:57.008361 (+ 27us) server_negotiation.cc:407] Connection header received
0812 03:12:57.008506 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:57.008509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:57.008579 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:57.008664 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:57.009299 (+ 635us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51288: BlockingRecv error: recv got EOF from 127.0.0.1:51288 (error 108)
Metrics: {"server-negotiator.queue_time_us":172}
W20260812 03:12:57.141093 19475 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:12:57.139687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51292 (local address 127.26.250.189:41353)
0812 03:12:57.139921 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:57.139925 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:57.140053 (+ 128us) server_negotiation.cc:407] Connection header received
0812 03:12:57.140126 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:57.140129 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:57.140283 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:57.140368 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:57.140951 (+ 583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51292: BlockingRecv error: recv got EOF from 127.0.0.1:51292 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
W20260812 03:12:57.400696 19475 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 03:12:57.399511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57640 (local address 127.26.250.189:41353)
0812 03:12:57.399663 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:57.399666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:57.399735 (+ 69us) server_negotiation.cc:407] Connection header received
0812 03:12:57.399874 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:57.399877 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:57.399926 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:57.400010 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 03:12:57.400522 (+ 512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57640: BlockingRecv error: recv got EOF from 127.0.0.1:57640 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 03:12:57.562567 19489 catalog_manager.cc:1259] Loaded cluster ID: a962f6387f45481b98c16b7fcb28233d
I20260812 03:12:57.562662 19489 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 420e374d01ec44f9a527b444de342b8b: loading cluster ID for follower catalog manager: success
I20260812 03:12:57.564450 19489 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 420e374d01ec44f9a527b444de342b8b: acquiring CA information for follower catalog manager: success
I20260812 03:12:57.565655 19489 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 420e374d01ec44f9a527b444de342b8b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:12:57.586280 19508 catalog_manager.cc:1259] Loaded cluster ID: a962f6387f45481b98c16b7fcb28233d
I20260812 03:12:57.586410 19508 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f66d123298b6409887bef8e582c56a39: loading cluster ID for follower catalog manager: success
I20260812 03:12:57.588201 19508 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f66d123298b6409887bef8e582c56a39: acquiring CA information for follower catalog manager: success
I20260812 03:12:57.589371 19508 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f66d123298b6409887bef8e582c56a39: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 03:12:57.919842 19613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:57.915102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57650 (local address 127.26.250.189:41353)
0812 03:12:57.915487 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:57.915491 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:57.915509 (+ 18us) server_negotiation.cc:407] Connection header received
0812 03:12:57.915567 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:57.915569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:57.915753 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:57.915870 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:12:57.916221 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:57.916890 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:57.917861 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:57.918139 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:57.918286 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:57.918687 (+ 401us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:12:57.919183 (+ 496us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:12:57.919298 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:57.919543 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 03:12:57.919639 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":90,"threads_started":1}
I20260812 03:12:57.927484 19620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:57.923303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60576 (local address 127.26.250.188:42319)
0812 03:12:57.923580 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:57.923584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:57.923599 (+ 15us) server_negotiation.cc:407] Connection header received
0812 03:12:57.923802 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:57.923806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:57.923945 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:57.924066 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:12:57.924355 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:57.924959 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:57.925830 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:57.926073 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:57.926175 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:57.926545 (+ 370us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:12:57.927008 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:12:57.927108 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:57.927200 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 03:12:57.927253 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":80,"threads_started":1}
I20260812 03:12:57.935295 19613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:57.931045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57666 (local address 127.26.250.189:41353)
0812 03:12:57.931206 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:57.931211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:57.931371 (+ 160us) server_negotiation.cc:407] Connection header received
0812 03:12:57.931568 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:57.931571 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:57.931639 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:57.931715 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:12:57.932029 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:57.932739 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:57.933983 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:57.934329 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:57.934458 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:57.934791 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:12:57.934878 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:12:57.934923 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:57.935080 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 03:12:57.935142 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 03:12:57.935894 19620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:57.931809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60580 (local address 127.26.250.188:42319)
0812 03:12:57.931978 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:57.931985 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:57.932039 (+ 54us) server_negotiation.cc:407] Connection header received
0812 03:12:57.932171 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:57.932174 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:57.932224 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:57.932298 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:12:57.932644 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:57.933417 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:57.934720 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:57.934960 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:57.935076 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:57.935340 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:12:57.935435 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:12:57.935509 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:57.935668 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 03:12:57.935734 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 03:12:57.937134 19629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 03:12:57.931148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55930 (local address 127.26.250.190:43041)
0812 03:12:57.931473 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0812 03:12:57.931478 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 03:12:57.931763 (+ 285us) server_negotiation.cc:407] Connection header received
0812 03:12:57.931881 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 03:12:57.931885 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 03:12:57.932073 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 03:12:57.932197 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 03:12:57.933572 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:57.934222 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:57.935386 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 03:12:57.935717 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 03:12:57.935864 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 03:12:57.936316 (+ 452us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 03:12:57.936725 (+ 409us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 03:12:57.936833 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 03:12:57.936910 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 03:12:57.936973 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":74,"threads_started":1}
I20260812 03:12:57.940847 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 19227
I20260812 03:12:57.950675 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 19312
I20260812 03:12:57.960006 27626 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskh6ikwC/build/debug/bin/kudu with pid 19394
2026-08-12T03:12:57Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3430 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6734 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (302676 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST